Summary of the invention
In view of above-mentioned, present disclose provides a kind of for distributing the method and device of stream data.Utilize this method and dress
It sets, the load of each data processing resources in future time window can be determined based on the feature of each data processing resources
Desired amount, and then determined based on load desired amount and current stream data to be allocated is assigned to each data processing resources
Allocation probability.
According to one aspect of the disclosure, a kind of method for distributing stream data is provided, comprising: obtain each stream
The current health degree of formula data processing resources in preceding health degree and is currently successfully measured;For each stream data process resource, base
In the current health degree of the stream data process resource and in preceding health degree, the present load of the stream data process resource is determined
Performance of expected change amount;Current successfully amount and present load performance of expected change amount based on each stream data process resource, determine institute
State the present load desired amount of each stream data process resource;Present load based on each stream data process resource
Desired amount determines the first data allocation probability of each stream data process resource;And it is distributed according to first data general
The stream data is distributed to the correspondence stream data process resource in each stream data process resource by rate.
Optionally, in one example, it is born in the current successfully amount based on each stream data process resource and currently
Performance of expected change amount is carried, before the present load desired amount for determining each stream data process resource, the method can be with
It include: the present load performance of expected change amount for adjusting each stream data process resource.Wherein, at each stream data
The sum of the present load performance of expected change amount adjusted for managing resource is zero, and present load performance of expected change amount is confirmed as increasing
Each stream data process resource present load performance of expected change amount adjusted be no more than each self-adjusting before present load
Performance of expected change amount.
Optionally, in one example, for each stream data process resource, based on the stream data process resource
Current health degree and in preceding health degree, determine the stream data process resource present load performance of expected change amount may include:
The current health degree of the stream data process resource is not less than in preceding health degree, and the present load performance of expected change amount is arranged
For positive value;Or be lower than in the current health degree of stream data processing load in preceding health degree, the present load is pre-
Phase variable quantity is set as negative value.
Optionally, in one example, the method can also include: each stream data process resource of acquisition preceding
Performance of expected change amount and in preceding load performance of expected change rate is loaded, and is directed to each stream data process resource, based on the streaming
The current health degree of data processing resources and in preceding health degree, determines that the present load variable quantity of the stream data process resource can
With include: current health degree for each stream data process resource, based on the stream data process resource, in preceding health degree
It is described in preceding performance of expected change amount and in preceding load performance of expected change rate, determine that the present load of the stream data process resource is pre-
Phase rate of change;And based on identified present load performance of expected change rate, determine the current of the stream data process resource
Load performance of expected change amount.
Optionally, in one example, the current health degree based on the stream data process resource, in preceding health degree and
In preceding load performance of expected change rate, determine the stream data process resource present load performance of expected change rate may include:
The current health degree of the stream data process resource is not less than in preceding health degree and described is no more than in preceding load performance of expected change amount
When first threshold, the present load performance of expected change rate is set greater than described in preceding load performance of expected change rate;At this
The current health degree of stream data process resource be not less than preceding health degree and it is described preceding load performance of expected change amount be more than described in
When first threshold, the present load performance of expected change rate is set smaller than described in preceding load performance of expected change rate;At this
The current health degree of stream data process resource be lower than preceding health degree and it is described it is preceding load performance of expected change amount absolute value not
When more than second threshold, the absolute value of the present load performance of expected change rate is set greater than described in the expected change of preceding load
Change the absolute value of rate;Or it is lower than in the current health degree of the stream data process resource in preceding health degree and described preceding negative
When carrying the absolute value of performance of expected change amount more than the second threshold, the absolute value of the present load performance of expected change rate is arranged
For less than the absolute value in preceding load performance of expected change rate.
Optionally, in one example, the method can also include: the user characteristics number for obtaining the stream data
According to;Based on the user characteristic data of the stream data, the streaming number is predicted using stream data process resource recommended models
According to the second data allocation probability for being directed to each stream data process resource;Based on the first data allocation probability and described
Two data allocation probabilitys determine third data allocation probability;And according to the third data allocation probability by the streaming number
According to the correspondence stream data process resource distributed in each stream data process resource.
According to another aspect of the present disclosure, it also provides a kind of for distributing the device of stream data, comprising: process resource is special
Acquiring unit is levied, is configured as obtaining the current health degree of each stream data process resource, in preceding health degree and current success
Amount;Performance of expected change amount determination unit is loaded, is configured as handling for each stream data process resource based on the stream data
The current health degree of resource and in preceding health degree, determines the present load performance of expected change amount of the stream data process resource;Load
Desired amount determination unit is configured as current successfully amount and the expected change of present load based on each stream data process resource
Change amount determines the present load desired amount of each stream data process resource;First data allocation probability determination unit, quilt
It is configured to the present load desired amount of each stream data process resource, determines each stream data process resource
First data allocation probability;And stream data allocation unit, being configured as will be described according to the first data allocation probability
Stream data distributes to the correspondence stream data process resource in each stream data process resource.
Optionally, in one example, described device can also include: load performance of expected change amount adjustment unit, be configured
To determine each stream in current successfully amount and present load performance of expected change amount based on each stream data process resource
Before the present load desired amount of formula data processing resources, the present load for adjusting each stream data process resource is expected
Variable quantity.Wherein, the sum of the present load performance of expected change amount adjusted of each stream data process resource is zero, and
Present load performance of expected change amount is confirmed as the expected change of present load adjusted of increased each stream data process resource
Change amount is no more than the present load performance of expected change amount before each self-adjusting.
Optionally, in one example, the load performance of expected change amount determination unit can be configured as: in the streaming number
It is not less than according to the current health degree of process resource in preceding health degree, sets positive value for the present load performance of expected change amount;
Or be lower than in the current health degree of stream data processing load in preceding health degree, by the present load performance of expected change amount
It is set as negative value.
Optionally, in one example, described device can also include: load performance of expected change data capture unit, be matched
Be set to obtain each stream data process resource in preceding load performance of expected change amount and in preceding load performance of expected change rate, and bear
Carrying performance of expected change amount determination unit includes: load performance of expected change rate determination module, is configured as at each stream data
Manage resource, current health degree based on the stream data process resource, preceding health degree, it is described preceding load performance of expected change amount with
And it is described in preceding load performance of expected change rate, determine the present load performance of expected change rate of the stream data process resource;And
Performance of expected change amount determining module is loaded, is configured as determining the streaming number based on identified present load performance of expected change rate
According to the present load performance of expected change amount of process resource.
Optionally, in one example, the load rate of change determining module can be configured as: in the stream data
The current health degree of process resource is not less than in preceding health degree and described when preceding load performance of expected change amount is no more than first threshold,
The present load performance of expected change rate is set greater than described in preceding load performance of expected change rate;In stream data processing
The current health degree of resource is not less than in preceding health degree and described when preceding load variation amount is more than the first threshold, will be described
Present load performance of expected change rate is set smaller than described in preceding load performance of expected change rate;In the stream data process resource
Current health degree be lower than preceding health degree and it is described it is preceding load performance of expected change amount absolute value be no more than second threshold when, by institute
The absolute value for stating present load performance of expected change rate is set greater than the absolute value in preceding load performance of expected change rate;Or
The stream data process resource current health degree be lower than preceding health degree and it is described it is preceding load performance of expected change amount it is absolute
When value is more than the second threshold, the absolute value of the present load performance of expected change rate is set smaller than described in preceding load
The absolute value of performance of expected change rate.
Optionally, in one example, described device can also include: user characteristic data acquiring unit, be configured as
Obtain the user characteristic data of the stream data;Second data allocation probability determination unit is configured as based on the streaming
The user characteristic data of data predicts that the stream data is directed to each streaming number using stream data process resource recommended models
According to the second data allocation probability of process resource;Third data allocation probability determination unit is configured as based on first number
According to allocation probability and the second data allocation probability, third data allocation probability is determined;And the stream data distribution list
Member is configured as: the stream data being distributed to each stream data processing money according to the third data allocation probability
Correspondence stream data process resource in source.
According to another aspect of the present disclosure, a kind of calculating equipment is also provided, comprising: at least one processor;And storage
Device, the memory store instruction, when described instruction is executed by least one described processor, so that described at least one
It manages device and executes method as described above.
According to another aspect of the present disclosure, a kind of machine readable storage medium is also provided, executable instruction is stored with, institute
State instruction makes the machine execute method as described above upon being performed.
Using disclosed method and device, determined by the feature based on each data processing resources in future time
The load performance of expected change amount of each data processing resources in window, and then determined based on loading performance of expected change amount and successfully measuring
The load desired amount of each data processing resources in future time window, be then based on load desired amount come determine will currently to point
The stream data matched is assigned to the allocation probability of each data processing resources, so as to according to the spy of each data processing resources
Sign is to distribute stream data, so as to realize the load balancing of each data processing resources.
Using disclosed method and device, by adjusting the present load performance of expected change of each stream data process resource
Amount, so that the sum of the present load performance of expected change amount adjusted of each stream data process resource is zero, and present load
The present load performance of expected change amount adjusted that performance of expected change amount is confirmed as increased each stream data process resource does not surpass
Present load performance of expected change amount before crossing each self-adjusting, so that all stream datas in next time window can
Corresponding stream data process resource is successfully distributed to, and the load that can be avoided each stream data process resource is more than
Processing capacity.
Using disclosed method and device, it is not less than by the current health degree in the stream data process resource preceding
When health degree, positive value, or the current health degree in stream data processing load are set by present load performance of expected change amount
Lower than in preceding health degree, negative value is set by present load performance of expected change amount, it can be in the health of stream data process resource
When degree is deteriorated, the anticipated load amount of the stream data process resource is reduced, and the health in stream data process resource improves
When, increase the load desired amount of the stream data process resource, thus it enables that the load dynamic of the stream data process resource
Ground is adjusted to realize load balancing.
Using disclosed method and device, by based on each stream data process resource in preceding load performance of expected change
Amount and present load performance of expected change rate is determined in preceding load performance of expected change rate, and based on present load performance of expected change rate
True present load performance of expected change amount can adjust stream data processing according to the real-time status of each stream data process resource
The load of resource.
Using disclosed method and device, by being based in preceding health degree, current health degree, in preceding load performance of expected change
Amount and present load performance of expected change rate is determined in preceding load performance of expected change rate, can not be deteriorated in health degree or healthy
When degree is deteriorated, the speed that rationally control load rises or falls, to realize smooth load balancing.
Using disclosed method and device, by predicting for corresponding stream data to be assigned to respectively based on user characteristic data
Second data allocation probability of a stream data process resource, be then based on the second data allocation probability and according to stream data at
The first data allocation probability that the real-time status of reason resource determines determines third data allocation probability, and then based on third data point
Distribute stream data with probability, so as in conjunction with stream data process resource feature and user characteristics realize that load is equal
Weighing apparatus.
Specific embodiment
Theme described herein is discussed below with reference to example embodiment.It should be understood that discussing these embodiments only
It is in order to enable those skilled in the art can better understand that being not to claim to realize theme described herein
Protection scope, applicability or the exemplary limitation illustrated in book.It can be in the protection scope for not departing from present disclosure
In the case of, the function and arrangement of the element discussed are changed.Each example can according to need, omit, substitute or
Add various processes or component.In addition, feature described in relatively some examples can also be combined in other examples.
As used in this article, term " includes " and its modification indicate open term, are meant that " including but not limited to ".
Term "based" indicates " being based at least partially on ".Term " one embodiment " and " embodiment " expression " at least one implementation
Example ".Term " another embodiment " expression " at least one other embodiment ".Term " first ", " second " etc. may refer to not
Same or identical object.Here may include other definition, either specific or implicit.Unless bright in context
It really indicates, otherwise the definition of a term is consistent throughout the specification.
The method and device for being used to distribute stream data of the disclosure is described presently in connection with attached drawing.
Fig. 1 is an exemplary structural block diagram of SiteServer LBS.As shown in Figure 1, in SiteServer LBS 100,
Including multiple stream data process resources 140, (although shown in Fig. 1 being 4, the quantity of stream data process resource can be with
It is arbitrary).Stream data process resource 140 can be by one or more data processing servers for calculating equipment and forming.
Multiple users can issue request of data via network 120 by user terminal 110.Network 120 can be cable network or wireless
Network.Pending data corresponding to the request of data of each user can distribute to multiple streamings by load balancing apparatus 130
Data processing resources 140, so that each stream data process resource completes the processing task for each pending data jointly,
To improve the response speed to the request of data of user.Load balancing apparatus 140 should make each stream data process resource
Load meets respective processing capacity as far as possible, to make full use of the processing capacity of each stream data process resource.
Fig. 1 shows an example of SiteServer LBS just for the sake of schematical explanation, and the disclosure can be with
SiteServer LBS suitable for other frameworks.
Fig. 2 is an exemplary configuration diagram of the applicable data processing system of the disclosure.As shown in Fig. 2, at data
Reason system 200 includes log acquisition apparatus 210, feature deriving means 220, trigger device 230, property data base 240, loads
Weigh device 250 and stream data process resource 260.
The data processing log information of 210 acquisition system of log acquisition apparatus may include each stream data processing money
Whether stream data handled by source processing result, is successfully processed, the corresponding user information of stream data, data processing time
Deng.Feature deriving means 220 can extract each stream data process resource based on the log collected of log acquisition apparatus 210
User characteristics corresponding to 260 characteristic and stream data processed.Feature deriving means 220 can use stream calculation
Processing engine (such as FLink, Spark etc.) is in real time processed log, to extract at user characteristics and stream data
Manage resource characteristic.Stream data process resource feature may include the health of each stream data process resource at a time
Degree, predetermined time window load capacity and successfully amount, successfully measure accounting (successfully amount accounts for the ratio of total load amount) etc., can be with
User characteristics including stream data handled by the stream data process resource.Stream data (or flow data) typically refers to one
Group sequence, a large amount of, data sequence that rapidly, continuously reaches, it is generally the case that data flow can be considered as one continue at any time and
The dynamic data set to increase without limitation.
The various characteristics that feature deriving means 220 extract can store in property data base 240.Trigger device
230 for triggering load balancing apparatus 250 to distribute stream data.Trigger device 230 can be using clocked flip (i.e. every pre-
Interval of fixing time triggers the distribution of data) mode trigger stream data distribution, be also based on stream to trigger.In addition,
Trigger device 230 can also the triggering stream data distribution when generating stream data to be allocated.
Load balancing apparatus 230 can be special based on the items in property data base 240 under the triggering of trigger device 230
Stream data is distributed to corresponding stream data process resource 260 by sign, with by stream data process resource 260 come convection type number
According to being handled.After completing stream data processing, generated log is acquired by log acquisition apparatus 210, new to generate
Daily record data.
Stream data distribution method in the embodiment of the present disclosure can be held by load balancing apparatus as depicted in figs. 1 and 2
Row.
Fig. 3 is the flow chart according to the stream data distribution method of one embodiment of the disclosure.Fig. 4 is for illustrating this
The schematic diagram of concept in the stream data distribution method of disclosed one embodiment.
As shown in figure 3, obtaining the current health degree of each stream data process resource in block 302, in preceding health degree and working as
It is preceding successfully to measure.Current health degree refer at current time based on each stream data process resource feature currently extracted and
Determining health degree, currently successfully amount refers in the predetermined time window (time window as shown in Figure 4 until current time
N each stream data process resource handles successful stream data amount in).It is the health determined at the preceding moment in preceding health degree
Degree.
Stream data is the chronologically-based data successively reached, and whenever having stream data arrival, needing should be to
The stream data of distribution is assigned to stream data process resource appropriate, to be handled.As shown in figure 4, stream data N and stream
Formula data N+1 is successively reached, and stream data N is in preceding assigned stream data, and stream data N+1 is current stream to be allocated
Formula data.However, due to each stream data process resource actual treatment ability may with contemplate that difference, thus be assigned to
The stream data of each stream data process resource may not can all successes processed.It is provided for the processing of each stream data
Source, it is the stream data process resource in the given time window that successful stream data summation is handled in given time window
Interior successful amount.
Under normal conditions, current time can be current stream data (such as stream data N+1 shown in Fig. 4) to be allocated
At the time of reaching load balancing apparatus, or it can be batch operation of the trigger device triggering to current stream data to be allocated
Moment.
After getting the characteristics such as current health degree, in block 304, for each stream data process resource, base
In the current health degree of the stream data process resource and in preceding health degree, the present load of the stream data process resource is determined
Performance of expected change amount.Present load performance of expected change amount refers to, in future time window, a certain stream data determined by current time
The stream data total amount that process resource also is expected to processing handles successful stream data total amount relative to actual time window
Variable quantity.Present load performance of expected change amount may be confirmed as increasing or decreasing, when being confirmed as increasing, it is contemplated that corresponding streaming
The present load desired amount of data resource is relative to currently successfully amount increased, when being confirmed as reducing, it is contemplated that corresponding
The present load desired amount of stream data resource is reduced relative to currently successfully measuring.According to each stream data process resource
Current health degree and in preceding health degree, future time window (time window N+1 as shown in Figure 4) can be determined relative to working as
The variable quantity for the stream data that preceding time window (as shown in figure 4, time window N) can be handled.For example, can be according to current strong
Kang Du determines present load performance of expected change amount relative to the variation tendency in preceding health degree.
Then, in block 306, current successfully amount and present load performance of expected change based on each stream data process resource
Amount, determines the present load desired amount of each stream data process resource.Present load desired amount refers to be determined at current time
It is contemplated that the stream data amount that future time window is capable of handling.It can work as on the basis of currently successfully measuring plus identified
Preceding load performance of expected change amount, to determine corresponding streaming data processing resources in the load desired amount of future time window.
It is determining, based on the present load desired amount of each stream data process resource, to determine each streaming in block 308
First data allocation probability of data processing resources.It in one example, can working as based on each stream data process resource
Preceding load desired amount determines the distribution that the stream data of each stream data process resource is distributed in future time window
Ratio.Since in future time window, in addition to current stream data N+1 to be allocated, other stream datas are divided
The corresponding streaming data processing resources of dispensing, thus the allocation proportion can be regarded as current stream data N+1 to be allocated will be by
Distribute to the first data allocation probability of each stream data process resource.
It in one example, can be in current successfully amount and present load based on each stream data process resource
Performance of expected change amount before the present load desired amount for determining each stream data process resource, adjusts each stream data processing
The present load performance of expected change amount of resource, so that the present load performance of expected change amount adjusted of each stream data process resource
The sum of be zero, and present load performance of expected change amount is confirmed as increased each the adjusted of stream data process resource and works as
Preceding load performance of expected change amount is no more than the present load performance of expected change amount before each self-adjusting.
The present load performance of expected change amount of each stream data process resource is determined based on respective health degree situation,
It thus may be uneven between the present load performance of expected change amount of each stream data process resource.A unbalanced example
It is the summation of decline cannot to be covered in the sum total that the stream data sendout of future time window rises, to will lead to portion
Shunt data cannot by the distribution of work, thus need to the present load performance of expected change amount of each stream data process resource into
Row adjustment.As an example, if the quantity of stream data process resource is three, i.e. A, B, C, it is assumed that the present load of A, B, C
Performance of expected change amount is followed successively by 10,20, -22 (positive value indicates to increase, and negative value indicates to reduce), then shows to increase in subsequent time A
The upper limit added is that 10, B can be with the increased upper limit for 20, and C should at least reduce 22.At this point, A, B, C present load adjusted are pre-
Phase variable quantity can be respectively the combinations such as 10,12, -22 or 2,20, -22, as long as load performance of expected change amount adjusted is respective
Above-mentioned limit value within, and guarantee to load increased quantity in each stream data process resource and can cover what load was reduced
Quantity.
In one example, health degree can be ranked up, and the best stream data of health degree is successively handled into money
The load performance of expected change amount in source is adjusted being capable of increased maximum value (the load performance of expected change amount determined i.e. in block 306) to it.
For example, if the health degree of A, B, C are followed successively by 0.9,0.88,0.44.Since the health degree of A is best, since C needs to reduce in total
22, thus the load performance of expected change amount of A can be adjusted to 10, the load performance of expected change amount of B is then adjusted to 12.Example again
Such as, it is assumed that tetra- stream data process resources of a total of A, B, C, D, the load performance of expected change amount determined in block 306 are followed successively by
10,20,10, -35, and be followed successively by 0.9,0.88,0.8,0.44 in the health degree of time window N, then after increasing by 10 for A,
Most preferably B thus can increase by 20 for B to health degree, then increase by 5 for C.
By above-mentioned adjustment process, it can guarantee that stream data to be allocated can be by the distribution of work, and can make full use of
The processing capacity of each stream data process resource.
After determining the first data allocation probability, in block 310, stream data is divided according to the first data allocation probability
Correspondence stream data process resource in each stream data process resource of dispensing.For example, can be by current streaming to be allocated
Data distribute to the highest stream data process resource of the first data allocation probability.
As a result, if whithin a period of time, the first data allocation probability of a certain stream data process resource highest always,
The stream data for then distributing to the stream data process resource can be relatively more.If distributing to the stream of the stream data process resource
Formula data volume has exceeded processing capacity, then its health degree is deteriorated, thus the load desired amount of the stream data process resource also can
It accordingly decreases, and then its first data allocation probability can also reduce, thus, it is possible to reduce to distribute to the stream data process resource
Stream data amount.On the other hand, if whithin a period of time, the first data distribution of a certain stream data process resource is general
Rate is lower always, then distributes to its stream data and can reduce, and then the health degree of the stream data process resource can rise, most
The the first data allocation probability determined for it is caused to rise eventually, so that distributing to the streaming number of the stream data process resource
Rise according to amount.Therefore, by the embodiment, streaming number can be dynamically adjusted according to the state of each stream data process resource
According to allocation proportion, to realize load balancing.
Fig. 5 is that the load performance of expected change amount in the stream data distribution method according to one embodiment of the disclosure determined
One exemplary flow chart of journey.
As shown in figure 5, for each stream data process resource, comparing current health degree and in preceding health in block 502
Degree.
If current health degree is not less than in preceding health degree, in block 504, the determination of present load performance of expected change amount is positive
Value.If current health degree is not less than in preceding health degree, show that the health degree of the stream data process resource is not deteriorated, in turn
Show that the processing capacity of the stream data process resource is not less than the stream data amount for being actually allocated to it.Thus it is expected
Future time window, the stream data process resource are capable of handling more stream datas, thus can be expected by present load
Variable quantity is determined as positive value.
If current health degree is lower than in preceding health degree, in block 506, present load performance of expected change amount is determined as negative value.
If current health degree is lower than in preceding health degree, show in the stream for distributing to the stream data process resource in preceding time window
Formula data volume has been more than its processing capacity.Therefore, it is possible to it is contemplated that future time window, it is contemplated that in future time window
The stream data amount that the stream data process resource can be handled can reduce.Present load performance of expected change amount can thus be determined
For negative value.
Fig. 6 is that the load performance of expected change amount in the stream data distribution method according to one embodiment of the disclosure determined
Another exemplary flow chart of journey.
As shown in fig. 6, in block 602, obtain each stream data process resource in preceding load performance of expected change amount and preceding
Load performance of expected change rate.Present load performance of expected change rate refer to current time determine load performance of expected change amount relative to
The rate of rise or fall off rate of load performance of expected change amount determined by the preceding moment.
It is obtaining after preceding load performance of expected change rate, is being based in block 604 for each stream data process resource
The current health degree of the stream data process resource is expected in preceding health degree, in preceding load performance of expected change amount and in preceding load
Rate of change determines the present load performance of expected change rate of the stream data process resource.
Then, in block 606, based on identified present load performance of expected change rate, the stream data process resource is determined
Present load performance of expected change amount.In one example, load performance of expected change amount can based on load desired amount accounting (that is,
Given time window, the total load that the load desired amount of the stream data process resource accounts for all stream data process resources are expected
The ratio of amount) variable quantity determine.And the variable quantity for loading desired amount accounting can be determined based on step-length is adjusted.It can be with needle
To each stream data process resource, determine that each step-length is corresponding based on the historic load amount of each stream data process resource
Baseline.For example, for stream data process resource A, B, C, if the historic load amount of three is 10 in given time period
Ten thousand, 120,000,180,000.Then for A, baseline can be determined that β=10/ [(10+12+18) * M], and M refers to maximum step-length, i.e.,
Maximum stream data allocation proportion (the i.e. history streaming number of the stream data process resource can be reached by most adjusting M step-length
According to allocation proportion).β is the corresponding allocation proportion of each step-length.For example, if history stream data allocation proportion is 25%, most
It is adjust 16 step-lengths adjustable to 25% more, then β=25%/16.If currently adjusting step-length is C, present load is expected
The variable quantity for measuring accounting is C* β.So, it is assumed that in the stream data total amount of time window N+1 be Q, then present load is expected becomes
Change amount is C* β * Q.
Fig. 7 is that the load performance of expected change rate in the stream data distribution method according to one embodiment of the disclosure determines
One exemplary flow chart of process.Fig. 8 is that showing for load performance of expected change amount determination process is determined using method shown in Fig. 7
Example property tendency chart.Slope of a curve loads performance of expected change rate in Fig. 8.
As shown in fig. 7, for each stream data process resource, comparing current health degree and in preceding health in block 702
Degree.
If current health degree is not less than in preceding health degree, in block 704, bearing preceding for the stream data process resource is judged
Carry whether performance of expected change amount is more than first threshold.
If present load performance of expected change rate set in block 706 more than first threshold in preceding load performance of expected change amount
It is set to and is less than in preceding load performance of expected change rate.If being more than first threshold in preceding load performance of expected change amount, show the streaming
The load capacity growth rate of data processing resources is too fast, may have been approached its processing capacity upper limit, thus should reduce growth speed
Degree.At this point it is possible to reduce present load performance of expected change rate.
If being less than first threshold in preceding load performance of expected change rate, in block 708, by present load performance of expected change speed
Rate is set greater than in preceding load performance of expected change rate.If present load performance of expected change rate is less than first threshold, table
The processing capacity of the bright stream data process resource can be improved load capacity and increase speed far beyond actual loading.
It, can be by whether adjusting step-length more than the first step-length when determining load performance of expected change amount based on step-length is adjusted
Threshold value judges whether present load performance of expected change amount is more than first threshold.As shown in figure 8, at I sections, the setting of the first step-length threshold value
It is 12, the adjusting step-length that each moment determines before time 4 is respectively less than 12, therefore at moment 1, moment 2, moment 3 and moment 4
Determining adjusting step-length is followed successively by 2,4,8,16, and the step-length increased numbers relative to previous moment are followed successively by 1,2,4,8.From Fig. 8
As can be seen that from moment 1 to the moment 4, the rate of rise for adjusting step-length rises, which is load performance of expected change amount at I sections
The rapid increase stage.And the step-length at moment 4 is confirmed as 16, has been more than 12, thus at the moment 5, it adjusts step-length and is determined 18, phase
2 are only increased for the moment 4, increasing degree is less than increasing degree of the moment 4 relative to the moment 3 and correspondingly works as at the moment 5
The growth recession of preceding load variation amount.II section in fig. 8 has been more than the first step-length threshold value 12 due to adjusting step-length,
The rate of rise for thus adjusting step-length slows down.
If current health degree be lower than in preceding health degree, in block 710, judge it is preceding load performance of expected change amount absolute value be
No is more than second threshold.
If being more than second threshold in the absolute value of preceding load performance of expected change amount, in block 712, by present load performance of expected change
Rate is set smaller than in preceding load performance of expected change rate.It is lower than in the case where preceding health degree in current health degree, if worked as
The absolute value of preceding load performance of expected change rate is more than second threshold, then shows reduction of speed under the load capacity of the stream data process resource
It spends fastly, is likely to result in processing capacity waste, thus need to reduce load capacity decrease speed.
If the absolute value in preceding load performance of expected change amount is less than second threshold, in block 614, by the expected change of present load
Change rate to be set greater than in preceding load performance of expected change rate.It is lower than in the case where preceding health degree in current health degree, if
The absolute value of present load performance of expected change rate is less than second threshold, then under the load capacity for showing the stream data process resource
Reduction of speed degree is slow, can accelerate decrease speed.
It, can be by whether adjusting the absolute value of step-length more than the when determining load performance of expected change amount based on step-length is adjusted
Two step-length threshold values judge whether present load performance of expected change amount is more than second threshold.As shown in figure 8, since the moment 13, when
Preceding health degree is lower than in preceding health degree, i.e. the health degree of the stream data process resource starts to be deteriorated.Thus since the moment 13
Adjusting step-length becomes negative value.At III sections, the second step-length threshold value is set as 12, from moment 13 to the moment 17, adjusts the absolute of step-length
Value is less than 12, thus adjusts step-length and be followed successively by -2, -4, -8, -16.Therefore it is determined at moment 1, moment 2, moment 3 and moment 4
The absolute value of adjusting step-length be followed successively by 1,2,4,8 relative to the increased numbers of the adjusting step-length absolute value of previous moment.From Fig. 8's
III sections as can be seen that the rate of rise for adjusting step-length absolute value at this stage rises.And the moment 17 determine adjusting step-length be-
16, absolute value has been more than 12, thus at the moment 18, adjusting step-length is confirmed as -18, and absolute value increases 2 relative to the moment 17,
Increasing degree lower than the moment 17 relative to the moment 16.III sections are the quick of adjusting step-length (i.e. load performance of expected change amount) in Fig. 8
Decline stage, IV sections are the slow decline stage (the slow decline stage of corresponding load variation amount) for adjusting step-length.
Fig. 9 is the exemplary diagram of the partial routine of stream data distribution method according to another embodiment of the present disclosure.
As shown in figure 9, obtaining the user characteristic data of stream data in block 902.User characteristic data may include using
Family behavioral data, history process record of stream data relevant to the user etc..
After getting user characteristic data, in block 904, the user characteristic data based on stream data utilizes stream
Formula data processing resources recommended models predict that stream data is directed to the second data allocation probability of each stream data process resource.
In the training process of stream data process resource recommended models, it can be based on stream data history process record, to training sample
User characteristic data sample in this carries out mark, to mark out history stream data corresponding to each user characteristic data sample
Process resource.That is, marking out the stream data processing money of stream data corresponding to processed specific user's characteristic sample
Source.It is then possible to which the sample marked using these is come training pattern.Second data allocation probability is predicted based on user characteristics
Recommendation probability out is able to reflect the corresponding stream data of different user and is suitble to by which stream data process resource
Reason.
After determining the second data allocation probability, in block 906, it is based on the first data allocation probability and the second data
Allocation probability determines third data allocation probability.In one example, the mode that can use weighted sum determines third data
Allocation probability.At this point it is possible to determine weight according to business demand.For example, if it is desired to data distribution is more concerned about user characteristics,
Then biggish weight can be set for the second data allocation probability, if it is desired to which data distribution increasingly focuses on each stream data money
Bigger weight can be arranged in the self character in source then for the first data allocation probability.
Then, in block 908, the stream data is distributed at each stream data according to third data allocation probability
Manage the correspondence stream data process resource in resource.For example, current stream data to be allocated can be distributed to current determination
Stream data process resource corresponding to third data allocation probability maximum value out.In another example, it can also compare and go through
The third data allocation probability that history each moment determines, if in allocation history, the third data of certain stream data process resource
Allocation probability continues highest, then stream data can be distributed to the high stream data processing money of third data allocation probability time
Source.
Figure 10 is the structural block diagram according to the stream data distributor of one embodiment of the disclosure.As shown in Figure 10,
Stream data distributor 1000 include process resource feature acquiring unit 1010, load performance of expected change amount determination unit 1020,
Load desired amount determination unit 1030, the first data allocation probability determination unit 1040 and stream data allocation unit 1050.
Process resource feature acquiring unit 1010 be configured as obtaining the current health degree of each stream data process resource,
It currently measures in preceding health degree and successfully.Load performance of expected change amount determination unit 1020 is configured as handling for each stream data
Resource, current health degree based on the stream data process resource and in preceding health degree determine the stream data process resource
Present load performance of expected change amount.
After determining present load performance of expected change amount, load desired amount determination unit 1030 is based at each stream data
Current successfully amount and the present load performance of expected change amount for managing resource, determine that the present load of each stream data process resource is pre-
Phase amount.It is pre- that first data allocation probability determination unit 1040 is configured as the present load based on each stream data process resource
Phase amount, determines the first data allocation probability of each stream data process resource.After determining the first data allocation probability, stream
Formula data allocation unit 1050 distributes to stream data in each stream data process resource according to the first data allocation probability
Correspondence stream data process resource.
Figure 11 is the structural block diagram of stream data distributor according to another embodiment of the present disclosure.Figure 12 is according to this
One exemplary structure of the load performance of expected change amount determination unit in the stream data distributor of disclosed one embodiment
Block diagram.As shown in figure 11, stream data distributor 1100 includes process resource feature acquiring unit 1110, load performance of expected change
Data capture unit 1120, load performance of expected change amount determination unit 1130, load desired amount determination unit 1140, the first data point
With probability determining unit 1150, user characteristic data acquiring unit 1160, the second data allocation probability determination unit 1170, third
Data allocation probability determination unit 1180 and stream data allocation unit 1190.
Process resource feature acquiring unit 1110 be configured as obtaining the current health degree of each stream data process resource,
It currently measures in preceding health degree and successfully.Performance of expected change data capture unit 1120 is loaded, is configured as obtaining each stream data
Process resource in preceding load performance of expected change amount and in preceding load rate of change.Then, performance of expected change amount determination unit is loaded
1130 determine that present load is expected based on acquired current health degree, in preceding health degree and in preceding load performance of expected change rate
Variable quantity.
As shown in figure 12, in this example, load performance of expected change amount determination unit includes that load performance of expected change rate determines
Module 1131 and load performance of expected change amount determining module 1132.Load performance of expected change rate determination module 1131 is configured as being directed to
Each stream data process resource, current health degree based on the stream data process resource become in preceding health degree, in preceding load
Change amount and in preceding load performance of expected change rate, determines the present load performance of expected change rate of the stream data process resource.It is negative
Performance of expected change amount determining module 1132 is carried to be configured as determining the streaming number based on identified present load performance of expected change rate
According to the present load performance of expected change amount of process resource.
In addition, stream data distributor can also include load performance of expected change amount adjustment unit (not shown).It is negative
Performance of expected change amount adjustment unit is carried to be configured as in the current successfully amount based on each stream data process resource and currently bearing
Performance of expected change amount is carried, before the present load desired amount for determining each stream data process resource, is adjusted at each stream data
Manage the present load performance of expected change amount of resource.Wherein, the present load adjusted of each stream data process resource is expected becomes
After the sum of change amount is zero, and present load performance of expected change amount is confirmed as the adjustment of increased each stream data process resource
Present load performance of expected change amount be no more than each self-adjusting before present load performance of expected change amount.
In one example, load rate of change determining module can be configured as: in the stream data process resource
Current health degree be not less than preceding health degree and it is described preceding load performance of expected change amount be no more than first threshold when, will it is described currently
Load performance of expected change rate is set greater than described in preceding load performance of expected change rate;In the current of the stream data process resource
Health degree be not less than preceding health degree and it is described preceding load variation amount be more than the first threshold when, the present load is pre-
Phase rate of change is set smaller than described in preceding load performance of expected change rate;In the current health degree of the stream data process resource
Lower than preceding health degree and it is described it is preceding load performance of expected change amount absolute value be no more than second threshold when, by the present load
The absolute value of performance of expected change rate is set greater than the absolute value in preceding load performance of expected change rate;Or in the streaming number
It is lower than according to the current health degree of process resource in preceding health degree and the absolute value in preceding load performance of expected change amount is more than described
When second threshold, the absolute value of the present load performance of expected change rate is set smaller than described in preceding load performance of expected change speed
The absolute value of rate.
After determining load performance of expected change amount, load desired amount determination unit 1140 is based on the processing of each stream data
Current successfully amount and the present load performance of expected change amount of resource, determine that the present load of each stream data process resource is expected
Amount.Then, the first present load desired amount of the data allocation probability determination unit 1140 based on each stream data process resource,
Determine the first data allocation probability of each stream data process resource.
User characteristic data acquiring unit 1160 is configured as obtaining the user characteristic data of stream data.Getting use
After the characteristic of family, 1170 user characteristic data based on stream data of the second data allocation probability determination unit utilizes stream
Formula data processing resources recommended models predict that stream data is directed to the second data allocation probability of each stream data process resource.
Then, third data allocation probability determination unit 1180 is based on the first data allocation probability and the second data allocation probability, determines
Third data allocation probability.
After determining third data allocation probability, stream data allocation unit is configured 1190 according to the third data
Stream data is distributed to the correspondence stream data process resource in each stream data process resource by allocation probability.
Above by reference to Fig. 1 to Figure 12, to the embodiment according to the method and device for distributing stream data of the disclosure
It is described.To the details mentioned in the description of embodiment of the method, it is being equally applicable to the reality of the device of the disclosure above
Apply example.
The device for distributing stream data of the disclosure can also use software or hardware using hardware realization
Combination with software is realized.All the embodiments in this specification are described in a progressive manner, between each embodiment
Same and similar part refers to each other.
The device for distributing stream data of the disclosure can also use software or hardware using hardware realization
Combination with software is realized.It taking software implementation as an example, is by equipment where it as the device on a logical meaning
Computer program instructions corresponding in memory are read into memory what operation was formed by processor.In the disclosure, for dividing
The device of oil-allocation type data, which for example can use, calculates equipment realization.
Figure 13 is the knot according to the calculating equipment for realizing stream data distribution method of one embodiment of the disclosure
Structure block diagram.As shown in figure 13, calculating equipment 1300 includes processor 1310, memory 1320, memory 1330, communication interface 1340
With internal bus 1350, and processor 1310, memory 1320, memory 1330, communication interface 1340 via bus 1350 connect
It is connected together.According to one embodiment, calculating equipment 1300 may include at least one processor 1310, at least one processing
It is computer-readable that device 1310 executes at least one of storage or coding in computer readable storage medium (that is, memory 1320)
It instructs (that is, above-mentioned element realized in a software form).
In one embodiment, computer executable instructions are stored in memory 1320, are made when implemented at least
One processor 1310: it obtains the current health degree of each stream data process resource, in preceding health degree and currently successfully measure;Needle
To each stream data process resource, current health degree based on the stream data process resource and in preceding health degree, determining should
The present load performance of expected change amount of stream data process resource;Current successfully amount based on each stream data process resource and
Present load performance of expected change amount determines the present load desired amount of each stream data process resource;Based on described each
The present load desired amount of stream data process resource determines the first data allocation probability of each stream data process resource;
And the stream data is distributed in each stream data process resource according to the first data allocation probability
Corresponding stream data process resource.
It should be understood that the computer executable instructions stored in memory 1320 make at least one processing when implemented
Device 1310 carries out the above various operations and functions described in conjunction with Fig. 1-12 in each embodiment of the disclosure.
According to one embodiment, a kind of program product of such as non-transitory machine readable media is provided.Non-transitory
Machine readable media can have instruction (that is, above-mentioned element realized in a software form), which when executed by a machine, makes
It obtains machine and executes the above various operations and functions described in conjunction with Fig. 1-12 in each embodiment of the disclosure.
Specifically, system or device equipped with readable storage medium storing program for executing can be provided, stored on the readable storage medium storing program for executing
Realize above-described embodiment in any embodiment function software program code, and make the system or device computer or
Processor reads and executes the instruction being stored in the readable storage medium storing program for executing.
In this case, it is real that any one of above-described embodiment can be achieved in the program code itself read from readable medium
The function of example is applied, therefore the readable storage medium storing program for executing of machine readable code and storage machine readable code constitutes of the invention one
Point.
The embodiment of readable storage medium storing program for executing include floppy disk, hard disk, magneto-optic disk, CD (such as CD-ROM, CD-R, CD-RW,
DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), tape, non-volatile memory card and ROM.It selectively, can be by communication network
Network download program code from server computer or on cloud.
It is above-mentioned that this specification specific embodiment is described.Other embodiments are in the scope of the appended claims
It is interior.In some cases, the movement recorded in detail in the claims or step can be come according to the sequence being different from embodiment
It executes and desired result still may be implemented.In addition, process depicted in the drawing not necessarily require show it is specific suitable
Sequence or consecutive order are just able to achieve desired result.In some embodiments, multitasking and parallel processing be also can
With or may be advantageous.
Not all step and unit are all necessary in above-mentioned each process and each system construction drawing, can be according to reality
Need to ignore certain step or units.Each step execution sequence be not it is fixed, can according to need and be determined.It is above-mentioned
Apparatus structure described in each embodiment can be physical structure, be also possible to logical construction, that is, some units may be by same
Physical entity is realized, is realized alternatively, some units may divide by multiple physical entities, alternatively, can be by multiple autonomous devices
Certain components realize jointly.
The term used in entire this specification " exemplary " means " being used as example, example or illustration ", and unexpectedly
Taste than other embodiments " preferably " or " have advantage ".For the purpose of the understanding to described technology is provided, it is embodied
Mode includes detail.However, it is possible to implement these technologies without these specific details.In some examples
In, in order to avoid the concept to described embodiment causes indigestion, well known construction and device is shown in block diagram form.
The optional embodiment of embodiment of the disclosure, still, the implementation of the disclosure is described in detail in conjunction with attached drawing above
Example be not limited to the above embodiment in detail, in the range of the technology design of embodiment of the disclosure, can to this
The technical solution of disclosed embodiment carries out a variety of simple variants, these simple variants belong to the protection of embodiment of the disclosure
Range.
The foregoing description of present disclosure is provided so that any those of ordinary skill in this field can be realized or make
Use present disclosure.To those skilled in the art, the various modifications carried out to present disclosure are apparent
, also, can also answer generic principles defined herein in the case where not departing from the protection scope of present disclosure
For other modifications.Therefore, present disclosure is not limited to examples described herein and design, but disclosed herein with meeting
Principle and novel features widest scope it is consistent.