CN110221917B

CN110221917B - Method and apparatus for distributing streaming data

Info

Publication number: CN110221917B
Application number: CN201910435829.7A
Authority: CN
Inventors: 吕赛霞; 翟毅腾; 马鸣; 欧阳威; 周扬; 吴彦伦
Original assignee: Advanced New Technologies Co Ltd
Current assignee: Advanced New Technologies Co Ltd; Advantageous New Technologies Co Ltd
Priority date: 2019-05-23
Filing date: 2019-05-23
Publication date: 2023-02-28
Anticipated expiration: 2039-05-23
Also published as: CN110221917A

Abstract

The present disclosure provides methods and apparatus for distributing streaming data. The method comprises the following steps: acquiring the current health degree, the previous health degree and the current success rate of each streaming data processing resource; for each streaming data processing resource, determining a current load expected variation of the streaming data processing resource based on a current health and a previous health of the streaming data processing resource; determining the current load expected quantity of each streaming data processing resource based on the current work load and the current load expected variable quantity of each streaming data processing resource; determining a first data allocation probability of each streaming data processing resource based on the current load expectation of each streaming data processing resource; and allocating the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the first data allocation probability.

Description

Method and apparatus for distributing streaming data

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to a method and apparatus for distributing streaming data.

Background

In a network system, it is often necessary to handle a large number of data requests from users. In order to respond to the data request of the user in time, a load balancing technology is required to be used. The data to be processed can be distributed to a plurality of data processing resources (such as data processing servers) in a balanced manner by utilizing the load balancing technology, so that the data processing tasks are completed by the data processing resources together. The load balancing technique in the prior art often results in too much data to be processed being allocated to a part of data processing resources, and thus the load of each data processing resource cannot be truly balanced.

Disclosure of Invention

In view of the foregoing, the present disclosure provides a method and apparatus for distributing streaming data. By using the method and the device, the expected load amount of each data processing resource in the next time window can be determined based on the characteristics of each data processing resource, and the distribution probability of distributing the streaming data to be distributed to each data processing resource is determined based on the expected load amount.

According to an aspect of the present disclosure, there is provided a method for distributing streaming data, comprising: acquiring the current health degree, the previous health degree and the current success rate of each streaming data processing resource; for each streaming data processing resource, determining a current load expected variation of the streaming data processing resource based on a current health and a previous health of the streaming data processing resource; determining the current load expected quantity of each streaming data processing resource based on the current work load and the current load expected variable quantity of each streaming data processing resource; determining a first data allocation probability of each streaming data processing resource based on the current load expectation of each streaming data processing resource; and allocating the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the first data allocation probability.

Optionally, in an example, before determining the current load expected amount of each streaming data processing resource based on the current work load of each streaming data processing resource and the current load expected change amount, the method may further include: adjusting the current load expected change of each streaming data processing resource. Wherein the sum of the adjusted current load expected change amounts of the streaming data processing resources is zero, and the current load expected change amount is determined to be that the adjusted current load expected change amount of each streaming data processing resource which is increased does not exceed the respective current load expected change amount before adjustment.

Optionally, in an example, determining, for each streaming data processing resource, a current expected amount of change in load for the streaming data processing resource based on the current health and the previous health of the streaming data processing resource may include: when the current health degree of the streaming data processing resource is not lower than the previous health degree, setting the current load expected variation to be a positive value; or when the current health degree of the streaming data processing load is lower than the previous health degree, setting the expected variation of the current load to be a negative value.

Optionally, in an example, the method may further include: obtaining the previous expected amount of change in load and the previous expected rate of change in load for each streaming data processing resource, and determining, for each streaming data processing resource, a current amount of change in load for the streaming data processing resource based on the current health and the previous health of the streaming data processing resource may include: for each streaming data processing resource, determining a current expected rate of change of load for the streaming data processing resource based on a current health of the streaming data processing resource, a previous health, the previous expected amount of change, and a previous expected rate of change of load; and determining a current expected amount of change in load for the streaming data processing resource based on the determined current expected rate of change in load.

Optionally, in one example, determining the current expected rate of change of load of the streaming data processing resource based on the current health, the previous health, and the previous expected rate of change of load may include: when the current health of the streaming data processing resource is not lower than a previous health and the previous expected load change amount does not exceed a first threshold, setting the current expected load change rate to be greater than the previous expected load change rate; setting the current load expected rate of change to be less than the previous load expected rate of change when the current health of the streaming data processing resource is not less than the previous health and the previous load expected amount of change exceeds the first threshold; setting an absolute value of the current load expected rate of change to be greater than an absolute value of the prior load expected rate of change when the current health of the streaming data processing resource is less than a prior health and the absolute value of the prior load expected amount of change does not exceed a second threshold; or when the current health of the streaming data processing resource is lower than the previous health and the absolute value of the expected change amount of the previous load exceeds the second threshold, setting the absolute value of the expected change rate of the current load to be smaller than the absolute value of the expected change rate of the previous load.

Optionally, in an example, the method may further include: acquiring user characteristic data of the streaming data; predicting a second data distribution probability of the streaming data aiming at each streaming data processing resource by utilizing a streaming data processing resource recommendation model based on the user characteristic data of the streaming data; determining a third data allocation probability based on the first data allocation probability and the second data allocation probability; and allocating the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the third data allocation probability.

According to another aspect of the present disclosure, there is also provided an apparatus for distributing streaming data, comprising: a processing resource characteristic obtaining unit configured to obtain a current health degree, a previous health degree and a current success amount of each streaming data processing resource; a load expected change determination unit configured to determine, for each streaming data processing resource, a current load expected change for the streaming data processing resource based on a current health and a previous health of the streaming data processing resource; a load expected amount determination unit configured to determine a current load expected amount of each streaming data processing resource based on a current work load of the streaming data processing resource and a current load expected variation amount; a first data allocation probability determining unit configured to determine a first data allocation probability of each streaming data processing resource based on a current load expected amount of the each streaming data processing resource; and a streaming data allocation unit configured to allocate the streaming data to a corresponding streaming data processing resource of the respective streaming data processing resources according to the first data allocation probability.

Optionally, in an example, the apparatus may further include: a load expected change amount adjusting unit configured to adjust a current load expected change amount of each streaming data processing resource before determining the current load expected amount of each streaming data processing resource based on the current work amount of each streaming data processing resource and the current load expected change amount. Wherein the sum of the adjusted current load expected change amounts of the streaming data processing resources is zero, and the current load expected change amount is determined to be that the adjusted current load expected change amount of each streaming data processing resource which is increased does not exceed the respective current load expected change amount before adjustment.

Optionally, in an example, the load expected change amount determining unit may be configured to: when the current health degree of the streaming data processing resource is not lower than the previous health degree, setting the current load expected variation to be a positive value; or when the current health degree of the streaming data processing load is lower than the previous health degree, setting the expected change amount of the current load to be a negative value.

Optionally, in an example, the apparatus may further include: a load expected change data acquisition unit configured to acquire a preceding load expected change amount and a preceding load expected change rate of each streaming data processing resource, and the load expected change amount determination unit includes: a load expected change rate determination module configured to determine, for each streaming data processing resource, a current load expected change rate for that streaming data processing resource based on a current health, a previous health, the previous load expected change amount, and the previous load expected change rate for that streaming data processing resource; and a load expected change determination module configured to determine a current load expected change of the streaming data processing resource based on the determined current load expected change rate.

Optionally, in one example, the load change rate determination module may be configured to: setting the current load expected rate of change to be greater than a previous load expected rate of change when a current health of the streaming data processing resource is not less than the previous health and the previous load expected amount of change does not exceed a first threshold; setting the current load expected rate of change to be less than the previous load expected rate of change when the current health of the streaming data processing resource is not less than the previous health and the previous load change amount exceeds the first threshold; setting an absolute value of the current load expected rate of change to be greater than an absolute value of the prior load expected rate of change when the current health of the streaming data processing resource is less than a prior health and the absolute value of the prior load expected amount of change does not exceed a second threshold; or when the current health of the streaming data processing resource is lower than the previous health and the absolute value of the expected change amount of the previous load exceeds the second threshold, setting the absolute value of the expected change rate of the current load to be smaller than the absolute value of the expected change rate of the previous load.

Optionally, in an example, the apparatus may further include: a user characteristic data acquisition unit configured to acquire user characteristic data of the streaming data; a second data allocation probability determination unit configured to predict a second data allocation probability of the streaming data for each streaming data processing resource by using a streaming data processing resource recommendation model based on user characteristic data of the streaming data; a third data allocation probability determination unit configured to determine a third data allocation probability based on the first data allocation probability and the second data allocation probability; and the streaming data distribution unit is configured to: and distributing the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the third data distribution probability.

According to another aspect of the present disclosure, there is also provided a computing device comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform the method as described above.

According to another aspect of the present disclosure, there is also provided a machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform the method as described above.

By using the method and the device disclosed by the invention, the expected load variation of each data processing resource in the next time window is determined based on the characteristics of each data processing resource, the expected load amount of each data processing resource in the next time window is further determined based on the expected load variation and the success amount, and then the distribution probability of distributing the current streaming data to be distributed to each data processing resource is determined based on the expected load amount, so that the streaming data can be distributed according to the characteristics of each data processing resource, and the load balance of each data processing resource is realized.

By using the method and apparatus disclosed herein, the sum of the adjusted current load expected variations of each streaming data processing resource is zero, and the adjusted current load expected variations of each streaming data processing resource, which are determined as increasing, do not exceed the respective current load expected variations before adjustment, by adjusting the current load expected variations of each streaming data processing resource, so that all streaming data in the next time window can be successfully allocated to the corresponding streaming data processing resource, and the load of each streaming data processing resource can be prevented from exceeding the processing capacity.

With the method and the device of the present disclosure, by setting the current load expected variation to a positive value when the current health degree of the streaming data processing resource is not lower than the previous health degree, or setting the current load expected variation to a negative value when the current health degree of the streaming data processing resource is lower than the previous health degree, the expected load capacity of the streaming data processing resource can be reduced when the health degree of the streaming data processing resource is degraded, and the load expected capacity of the streaming data processing resource can be increased when the health degree of the streaming data processing resource is better, thereby enabling the load of the streaming data processing resource to be dynamically adjusted to achieve load balancing.

With the method and apparatus of the present disclosure, the load of each streaming data processing resource can be adjusted according to the real-time status of the streaming data processing resource by determining the current load expected change rate based on the previous load expected change amount and the previous load expected change rate of the streaming data processing resource, and determining the current load expected change amount based on the current load expected change rate.

With the method and the device disclosed by the invention, by determining the current load expected change rate based on the previous health degree, the current health degree, the previous load expected change amount and the previous load expected change rate, the speed of load rising or falling can be reasonably controlled when the health degree is not deteriorated or the health degree is deteriorated, so that smooth load balance is realized.

By using the method and the device disclosed by the invention, the second data distribution probability of distributing the corresponding streaming data to each streaming data processing resource is predicted based on the user characteristic data, then the third data distribution probability is determined based on the second data distribution probability and the first data distribution probability determined according to the real-time state of the streaming data processing resource, and then the streaming data is distributed based on the third data distribution probability, so that the load balance can be realized by combining the characteristics of the streaming data processing resource and the user characteristics.

Drawings

A further understanding of the nature and advantages of the present disclosure may be realized by reference to the following drawings. In the drawings, similar components or features may have the same reference numerals. The accompanying drawings, which are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the detailed description serve to explain the embodiments of the disclosure without limiting the embodiments of the disclosure. In the drawings:

FIG. 1 is a block diagram of an example of a load balancing system;

FIG. 2 is an architectural diagram of one example of a data processing system to which the present disclosure is applicable;

FIG. 3 is a flow diagram of a streaming data distribution method according to one embodiment of the present disclosure;

fig. 4 is a schematic diagram for explaining a concept in a streaming data distribution method of an embodiment of the present disclosure;

fig. 5 is a flowchart of one example of a load expected change amount determination process in a streaming data distribution method according to one embodiment of the present disclosure;

fig. 6 is a flowchart of another example of a load expected change amount determination process in a streaming data distribution method according to an embodiment of the present disclosure;

fig. 7 is a flowchart of one example of a load expected change rate determination process in a streaming data distribution method according to one embodiment of the present disclosure;

FIG. 8 is an exemplary trend graph of a process for determining an expected amount of change in load using the method shown in FIG. 7;

fig. 9 is an exemplary diagram of a partial process of a streaming data distribution method according to another embodiment of the present disclosure;

fig. 10 is a block diagram of a streaming data distribution apparatus according to an embodiment of the present disclosure;

fig. 11 is a block diagram of a streaming data distribution apparatus according to another embodiment of the present disclosure;

fig. 12 is a block diagram of a configuration of an example of a load expected change amount determining unit in a streaming data distribution apparatus according to an embodiment of the present disclosure;

fig. 13 is a block diagram of a computing device for implementing a streaming data distribution method according to one embodiment of the present disclosure.

Detailed Description

The subject matter described herein will be discussed with reference to example embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and thereby implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as needed. In addition, features described with respect to some examples may also be combined in other examples.

As used herein, the term "include" and its variants mean open-ended terms in the sense of "including, but not limited to. The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first," "second," and the like may refer to different or the same object. Other definitions, whether explicit or implicit, may be included below. The definition of a term is consistent throughout the specification unless the context clearly dictates otherwise.

The method and apparatus for distributing streaming data of the present disclosure will now be described with reference to the accompanying drawings.

Fig. 1 is a block diagram of an example of a load balancing system. As shown in fig. 1, in the load balancing system 100, a plurality of streaming data processing resources 140 are included (although 4 are shown in fig. 1, the number of streaming data processing resources may be arbitrary). The streaming data processing resource 140 may be a data processing server comprised of one or more computing devices. Multiple users may issue data requests through user terminal 110 via network 120. The network 120 may be a wired network or a wireless network. The to-be-processed data corresponding to the data request of each user may be allocated to the plurality of streaming data processing resources 140 by the load balancing device 130, so that the streaming data processing resources collectively complete the processing task for each to-be-processed data, thereby improving the response speed of the data request of the user. The load balancing device 140 should make the load of each streaming data processing resource conform to the respective processing capacity as much as possible so as to fully utilize the processing capacity of each streaming data processing resource.

Fig. 1 shows an example of a load balancing system for illustrative purposes only, and the present disclosure may also be applied to load balancing systems of other architectures.

FIG. 2 is an architectural diagram of one example of a data processing system to which the present disclosure is applicable. As shown in fig. 2, the data processing system 200 includes a log collection device 210, a feature extraction device 220, a trigger device 230, a feature database 240, a load balancing device 250, and a streaming data processing resource 260.

The log collecting device 210 collects log information of data processing of the system, which may include streaming data processed by each streaming data processing resource, processing result, whether the processing is successful, user information corresponding to the streaming data, data processing time, and the like. The feature extraction device 220 may extract feature data of each streaming data processing resource 260 and a user feature corresponding to the processed streaming data based on the log collected by the log collection device 210. Feature extraction mechanism 220 may process the log in real-time using a stream computing processing engine (e.g., FLink, spark, etc.) to extract user features and streaming data processing resource features. The characteristics of the streaming data processing resources may include health degree of each streaming data processing resource at a certain time, load amount and success amount in a predetermined time window, success amount ratio (ratio of success amount to total load amount), and the like, and may also include user characteristics of the streaming data processed by the streaming data processing resources. Streaming data (or streaming data) generally refers to a set of sequential, massive, fast, continuous arriving data sequences, and in general, a data stream can be viewed as a dynamic data set that grows indefinitely as time goes on.

The various feature data extracted by the feature extraction means 220 may be stored in the feature database 240. The triggering device 230 is used to trigger the load balancing device 250 to distribute the streaming data. The triggering device 230 may trigger the streaming data allocation in a timed manner (i.e., trigger the data allocation once every predetermined time interval), and may also trigger on a streaming basis. Furthermore, the triggering device 230 may also trigger streaming data allocation when streaming data to be allocated is generated.

The load balancing device 230 may, under the trigger of the trigger device 230, allocate the streaming data to the corresponding streaming data processing resource 260 based on the features in the feature database 240, so that the streaming data is processed by the streaming data processing resource 260. After the streaming data processing is completed, the generated log is collected by the log collection means 210 to generate new log data.

The streaming data distribution method in the embodiment of the present disclosure may be performed by a load balancing apparatus as shown in fig. 1 and fig. 2.

Fig. 3 is a flow chart of a streaming data distribution method according to one embodiment of the present disclosure. Fig. 4 is a schematic diagram for explaining a concept in the streaming data distribution method of one embodiment of the present disclosure.

As shown in fig. 3, at block 302, a current health, a previous health, and a current amount of work for each streaming data processing resource are obtained. The current health degree refers to the health degree determined based on the characteristics of each streaming data processing resource extracted currently at the current time, and the current success amount refers to the streaming data amount successfully processed by each streaming data processing resource within a predetermined time window (such as time window N shown in fig. 4) from the current time. The previous health degree is a health degree determined at a previous time.

The streaming data is data that arrives sequentially in chronological order, and each time streaming data arrives, the streaming data to be allocated needs to be allocated to an appropriate streaming data processing resource and processed. As shown in fig. 4, streaming data N and streaming data N +1 arrive in sequence, where streaming data N is streaming data allocated before, and streaming data N +1 is streaming data to be allocated currently. However, since the actual processing capacity of each streaming data processing resource may differ from that expected, the streaming data allocated to each streaming data processing resource may not be able to be processed successfully in its entirety. For each streaming data processing resource, the sum of the streaming data successfully processed in a given time window is the amount of work done by the streaming data processing resource in the given time window.

In general, the current time may be a time when the streaming data to be currently distributed (for example, the streaming data N +1 shown in fig. 4) arrives at the load balancing apparatus, or may be a time when the triggering apparatus triggers the distribution operation on the streaming data to be currently distributed.

After the characteristic data, such as the current health, is obtained, at block 304, for each streaming data processing resource, the expected amount of change in the current load of the streaming data processing resource is determined based on the current health and the previous health of the streaming data processing resource. The current load expected variation is a variation of a total amount of streaming data that a certain streaming data processing resource expects to be able to process in a next time window determined at the current time, relative to a total amount of streaming data that the processing in the current time window succeeds. The current expected amount of change in load may be determined to increase or decrease, and when determined to increase, the current expected amount of load of the corresponding streaming data resource is expected to increase relative to the current amount of work, and when determined to decrease, the current expected amount of load of the corresponding streaming data resource is expected to decrease relative to the current amount of work. Based on the current health and the previous health of each streaming data processing resource, the amount of change in streaming data that can be processed in the next time window (e.g., time window N +1 shown in fig. 4) relative to the current time window (e.g., time window N shown in fig. 4) can be determined. For example, the expected amount of change in the current load may be determined according to a trend of the current health degree relative to the previous health degree.

Then, at block 306, a current expected amount of load for each streaming data processing resource is determined based on the current amount of work done for each streaming data processing resource and the current expected amount of change in load. The current load expectation amount refers to the amount of streaming data that is determined to be expected to be able to be processed in the next time window at the current time. The determined current expected amount of change in load may be added to the current amount of work to determine an expected amount of load for the corresponding streaming data processing resource for the next time window.

At decision block 308, a first data allocation probability for each streaming data processing resource is determined based on a current expected amount of load for each streaming data processing resource. In one example, the allocation ratio of the streaming data allocated to each streaming data processing resource in the next time window may be determined based on the current load expected amount of each streaming data processing resource. Since in the next time window, other streaming data than the streaming data N +1 to be currently allocated are already allocated to the corresponding streaming data processing resources, the allocation ratio can be regarded as the first data allocation probability that the streaming data N +1 to be currently allocated will be allocated to each streaming data processing resource.

In one example, the current expected amount of change in load for each streaming data processing resource may be adjusted before determining the current expected amount of change in load for each streaming data processing resource based on the current amount of work done for each streaming data processing resource and the current expected amount of change in load, such that the sum of the adjusted current expected amounts of change in load for each streaming data processing resource is zero and the adjusted current expected amount of change in load for each streaming data processing resource that is determined to be added does not exceed the respective pre-adjusted current expected amount of change in load.

The expected amount of change in the current load of each streaming data processing resource is determined based on the respective health condition, and thus there may be an imbalance between the expected amounts of change in the current load of each streaming data processing resource. An example of imbalance is that the aggregate of the rise in the distribution of streaming data in the next time window cannot cover the sum of the fall, which may result in some parts of the streaming data being unsuccessfully distributed, and thus requires adjustment of the expected amount of change in the current load of each streaming data processing resource. As an example, if the number of streaming data processing resources is three, i.e. a, B, C, assuming that the expected variation of the current load of a, B, C is 10, 20, -22 in order (positive values indicate increase and negative values indicate decrease), it indicates that at the next moment a can be increased by 10, B can be increased by 20, and C should be decreased by at least 22. In this case, the adjusted current expected load variation amounts of a, B, and C may be 10, 12, -22, or 2, 20, -22, respectively, as long as the adjusted expected load variation amounts are within the respective above-mentioned limits and the amount of load increase in each streaming data processing resource is ensured to cover the amount of load decrease.

In one example, the health may be ranked and the expected change in load for the best health streaming data processing resource may be adjusted in turn to the maximum value it can increase (i.e., the expected change in load determined in block 306). For example, if the health degrees of A, B and C are 0.9, 0.88 and 0.44 in sequence. Since a is the best health, since C needs to be reduced by 22 in total, the expected load variation of a can be adjusted to 10, and then the expected load variation of B can be adjusted to 12. As another example, assuming a total of four streaming data processing resources a, B, C, D, the expected variance of the load determined in block 306 is 10, 20, 10, -35 in order, and the health level in time window N is 0.9, 0.88, 0.8, 0.44 in order, then after adding 10 to a, the health level is best that B can be added 20 to B and then 5 to C.

Through the adjustment process, the streaming data to be allocated can be successfully allocated, and the processing capacity of each streaming data processing resource can be fully utilized.

After the first data allocation probability is determined, at block 310, the streaming data is allocated to a corresponding streaming data processing resource of the streaming data processing resources according to the first data allocation probability. For example, the streaming data to be currently allocated may be allocated to the streaming data processing resource with the highest first data allocation probability.

Thus, if the first data allocation probability of a streaming data processing resource is always the highest over a period of time, then a relatively large amount of streaming data will be allocated to that streaming data processing resource. If the amount of streaming data allocated to the streaming data processing resource exceeds the processing capacity, the health of the streaming data processing resource is deteriorated, and accordingly, the expected load amount of the streaming data processing resource is also decreased, and further, the first data allocation probability is also decreased, so that the amount of streaming data allocated to the streaming data processing resource can be decreased. On the other hand, if the first data allocation probability of a certain streaming data processing resource is always low in a period of time, the streaming data allocated to the streaming data processing resource is reduced, and the health degree of the streaming data processing resource is increased, which finally causes the first data allocation probability determined for the streaming data processing resource to be increased, so that the amount of the streaming data allocated to the streaming data processing resource is increased. Therefore, with this embodiment, the allocation ratio of the streaming data can be dynamically adjusted according to the status of each streaming data processing resource to achieve load balancing.

Fig. 5 is a flowchart of one example of a load expected change amount determination process in a streaming data distribution method according to one embodiment of the present disclosure.

As shown in FIG. 5, at block 502, the current health and the previous health are compared for each streaming data processing resource.

If the current health is not less than the previous health, the current load expected change is determined to be a positive value at block 504. If the current health degree is not lower than the previous health degree, the health degree of the streaming data processing resource is not deteriorated, and the processing capacity of the streaming data processing resource is further indicated to be not lower than the streaming data amount actually allocated to the streaming data processing resource. It is thus expected that at the next time window the streaming data processing resource is able to process more streaming data, and thus the current load expected change amount may be determined to be positive.

If the current health is lower than the previous health, the current load expected change amount is determined to be a negative value at block 506. If the current health is less than the previous health, then it is an indication that the amount of streaming data allocated to the streaming data processing resource in the previous time window has exceeded its processing capacity. Thus, the next time window can be expected, and the amount of streaming data that can be processed by the streaming data processing resource can be expected to decrease in the next time window. The current load expected change amount may thus be determined to be a negative value.

Fig. 6 is a flowchart of another example of a load expected change amount determination process in a streaming data distribution method according to an embodiment of the present disclosure.

As shown in fig. 6, at block 602, a prior expected amount of change in load and a prior expected rate of change in load for each streaming data processing resource are obtained. The current expected load change rate refers to an increase rate or a decrease rate of the expected load change amount determined at the current time with respect to the expected load change amount determined at the previous time.

After obtaining the previous expected rate of change in load, at block 604, for each streaming data processing resource, a current expected rate of change in load for that streaming data processing resource is determined based on the current health, the previous expected amount of change in load, and the previous expected rate of change in load for that streaming data processing resource.

Then, at block 606, a current expected amount of change in load for the streaming data processing resource is determined based on the determined current expected rate of change in load. In one example, the expected amount of change in load may be determined based on an expected amount of change in load over a ratio (i.e., a ratio of the expected amount of load for the streaming data processing resource to a total expected amount of load for all streaming data processing resources over a given time window). And the amount of change in the load expectation ratio may be determined based on the adjustment step size. A baseline for each step size may be determined for each streaming data processing resource based on the historical load amount of the respective streaming data processing resource. For example, for streaming data processing resources a, B, C, if the historical load of three is 10, 12, 18 million over a given time period. Then for a, its baseline may be determined as β =10/[ (10 +12+ 18) × M ], where M refers to the maximum step size, i.e. adjusting up to M steps enables reaching the maximum streaming data allocation ratio (i.e. historical streaming data allocation ratio) of the streaming data processing resource. And beta is the corresponding distribution proportion of each step. For example, if the historical streaming data distribution ratio is 25%, and up to 16 steps can be adjusted to 25%, then β =25%/16. And if the current adjusting step size is C, the change amount of the current load expected quantity ratio is C beta. Then, assuming that the total amount of streaming data in the time window N +1 is Q, the expected variation of the current load is C × β × Q.

Fig. 7 is a flowchart of one example of a process for determining a load expected rate of change in a streaming data distribution method according to one embodiment of the present disclosure. Fig. 8 is an exemplary trend graph of a process for determining an expected amount of change in load using the method shown in fig. 7. The slope of the curve in fig. 8 is the expected rate of change of the load.

As shown in FIG. 7, at block 702, the current health and the previous health are compared for each streaming data processing resource.

If the current health is not less than the previous health, at block 704, a determination is made as to whether the previous load expected change of the streaming data processing resource exceeds a first threshold.

If the previous expected load change amount exceeds the first threshold, then at block 706, the current expected load change rate is set to be less than the previous expected load change rate. If the expected amount of change in the previous load exceeds the first threshold, it indicates that the load amount of the streaming data processing resource has increased too fast, and may have approached the upper limit of the processing capacity thereof, and thus the increase rate should be decreased. At this point, the current load expected rate of change may be reduced.

If the previous expected rate of change in load does not exceed the first threshold, then at block 708, the current expected rate of change in load is set to be greater than the previous expected rate of change in load. If the current load expected change rate does not exceed the first threshold, the processing capacity of the streaming data processing resource is far beyond the actual load, and the load increase speed can be increased.

When determining the expected load change amount based on the adjustment step size, it may be determined whether the current expected load change amount exceeds a first threshold by whether the adjustment step size exceeds a first step size threshold. As shown in fig. 8, in the section I, the first step threshold is set to 12, and the adjustment steps determined at the respective times before the time 4 are all smaller than 12, so that the adjustment steps determined at the

times

1, 2, 3 and 4 are sequentially 2, 4, 8 and 16, and the number of step increases with respect to the previous time is sequentially 1, 2, 4 and 8. As can be seen from fig. 8, in the I-segment, from time 1 to time 4, the increase rate of the adjustment step increases, which is a rapid increase phase of the expected amount of change in the load. Whereas the step size at time 4 is determined to be 16, exceeding 12, so that at time 5 the adjustment step size is determined 18, increased by only 2 relative to time 4, by a magnitude that is less than the magnitude of the increase at time 4 relative to time 3, and correspondingly, the rate of increase of the current load change at time 5 decreases. In section II of fig. 8, since the adjustment step has exceeded the first step threshold 12, the increase rate of the adjustment step is slowed down.

If the current health is less than the previous health, at block 710, it is determined whether the absolute value of the expected amount of change in the previous load exceeds a second threshold.

If the absolute value of the previous expected amount of change in load exceeds the second threshold, the current expected rate of change in load is set to be less than the previous expected rate of change in load at block 712. In the case that the current health degree is lower than the previous health degree, if the absolute value of the expected change rate of the current load exceeds the second threshold, it indicates that the load amount of the streaming data processing resource is decreased too fast, which may cause a waste of processing capacity, and thus the load amount decrease rate needs to be decreased.

If the absolute value of the previous expected amount of load change does not exceed the second threshold, the current expected rate of load change is set to be greater than the previous expected rate of load change at block 614. And under the condition that the current health degree is lower than the previous health degree, if the absolute value of the current load expected change rate does not exceed the second threshold, the load capacity of the streaming data processing resource is reduced slowly, and the reduction speed can be increased.

When the expected load change amount is determined based on the adjustment step size, whether the current expected load change amount exceeds the second threshold may be determined by whether the absolute value of the adjustment step size exceeds the second threshold. As shown in fig. 8, from time 13, the current health is lower than the previous health, i.e., the health of the streaming data processing resource begins to deteriorate. The adjustment step thus becomes negative starting from instant 13. In paragraph III, the second step size threshold is set to 12, and from time 13 to time 17, the absolute value of the adjustment step size does not exceed 12, and thus the adjustment step sizes are-2, -4, -8, -16 in order. The absolute value of the adjustment step determined at time 1, time 2, time 3 and time 4 is thus increased by a

number

1, 2, 4, 8 in sequence relative to the absolute value of the adjustment step at the previous time. As can be seen from the section III of fig. 8, the rate of increase of the absolute value of the adjustment step increases at this stage. Whereas the adjustment step determined at the instant 17 is-16 and the absolute value exceeds 12, so that at the instant 18 the adjustment step is determined to be-18 and the absolute value is increased by 2 relative to the instant 17, below the magnitude of the increase of the instant 17 relative to the instant 16. In fig. 8, the section III is a fast decreasing stage of the adjustment step (i.e. the expected load change), and the section IV is a slow decreasing stage of the adjustment step (corresponding to the slow decreasing stage of the load change).

Fig. 9 is an exemplary diagram of a partial process of a streaming data distribution method according to another embodiment of the present disclosure.

As shown in fig. 9, in block 902, user characteristic data of streaming data is obtained. The user characteristic data may include user behavior data, historical processing records of streaming data associated with the user, and the like.

After the user characteristic data is obtained, in block 904, a second data allocation probability of the streaming data for each streaming data processing resource is predicted using the streaming data processing resource recommendation model based on the user characteristic data of the streaming data. In the training process of the streaming data processing resource recommendation model, marking can be performed on the user characteristic data samples in the training samples based on the streaming data historical processing records so as to mark the historical streaming data processing resources corresponding to the user characteristic data samples. That is, the streaming data processing resources of the streaming data corresponding to the processed specific user characteristic data sample are marked. These labeled samples can then be utilized to train the model. The second data allocation probability is a recommendation probability predicted based on the user characteristics, and can reflect which streaming data processing resource the streaming data corresponding to different users are suitable for processing.

After the second data allocation probability is determined, a third data allocation probability is determined based on the first data allocation probability and the second data allocation probability in block 906. In one example, the third data allocation probability may be determined by a weighted sum. At this time, the weight may be determined according to the service requirement. For example, if the data allocation is expected to focus more on the user characteristics, a greater weight may be set for the second data allocation probability, and if the data allocation is expected to focus more on the intrinsic characteristics of each streaming data resource, a greater weight may be set for the first data allocation probability.

Then, in block 908, the streaming data is allocated to a corresponding streaming data processing resource of the respective streaming data processing resources according to the third data allocation probability. For example, the streaming data to be currently allocated may be allocated to the streaming data processing resource corresponding to the currently determined maximum value of the third data allocation probability. In another example, the third data allocation probabilities determined at various times in the history may also be compared, and if the third data allocation probability of a certain streaming data processing resource continues to be the highest in the allocation history, the streaming data may be allocated to the streaming data processing resource with the next highest third data allocation probability.

Fig. 10 is a block diagram of a streaming data distribution apparatus according to an embodiment of the present disclosure. As shown in fig. 10, the streaming data distribution apparatus 1000 includes a processing resource characteristic acquisition unit 1010, a load expected change amount determination unit 1020, a load expected amount determination unit 1030, a first data distribution probability determination unit 1040, and a streaming data distribution unit 1050.

The processing resource characteristic acquisition unit 1010 is configured to acquire a current health degree, a previous health degree, and a current work amount of each streaming data processing resource. The expected amount of change in load determining unit 1020 is configured to determine, for each streaming data processing resource, a current expected amount of change in load for the streaming data processing resource based on the current health and the previous health of the streaming data processing resource.

After determining the current expected amount of change in load, the expected amount of load determining unit 1030 determines the current expected amount of load for each streaming data processing resource based on the current amount of work done for each streaming data processing resource and the current expected amount of change in load. The first data allocation probability determination unit 1040 is configured to determine a first data allocation probability for each streaming data processing resource based on a current expected load amount for each streaming data processing resource. After determining the first data allocation probability, the streaming data allocation unit 1050 allocates the streaming data to the corresponding streaming data processing resource in each streaming data processing resource according to the first data allocation probability.

Fig. 11 is a block diagram of a streaming data distribution apparatus according to another embodiment of the present disclosure. Fig. 12 is a block diagram of a configuration of an example of a load expected change amount determination unit in a streaming data distribution apparatus according to an embodiment of the present disclosure. As shown in fig. 11, the streaming data allocation apparatus 1100 includes a processing resource characteristic acquisition unit 1110, a load expected change data acquisition unit 1120, a load expected change amount determination unit 1130, a load expected amount determination unit 1140, a first data allocation probability determination unit 1150, a user characteristic data acquisition unit 1160, a second data allocation probability determination unit 1170, a third data allocation probability determination unit 1180, and a streaming data allocation unit 1190.

The processing resource feature obtaining unit 1110 is configured to obtain a current health degree, a previous health degree, and a current success amount of each streaming data processing resource. A load expected change data obtaining unit 1120 configured to obtain a previous load expected change amount and a previous load change rate of each streaming data processing resource. Then, the load expected change amount determination unit 1130 determines the current load expected change amount based on the acquired current health degree, previous health degree, and previous load expected change rate.

As shown in fig. 12, in this example, the load expected change amount determination unit includes a load expected change rate determination module 1131 and a load expected change amount determination module 1132. The load expected rate of change determination module 1131 is configured to determine, for each streaming data processing resource, a current load expected rate of change for the streaming data processing resource based on the current health, the previous amount of load change, and the previous load expected rate of change for the streaming data processing resource. The load expected change determination module 1132 is configured to determine a current expected change in load for the streaming data processing resource based on the determined current expected rate of change in load.

Furthermore, the streaming data distribution apparatus may further include a load expected change amount adjustment unit (not shown in the figure). The load expected change amount adjustment unit is configured to adjust the current load expected change amount of each streaming data processing resource before determining the current load expected amount of each streaming data processing resource based on the current work load of each streaming data processing resource and the current load expected change amount. Wherein the sum of the adjusted current load expected change amounts of the streaming data processing resources is zero, and the adjusted current load expected change amount of the streaming data processing resources, which is determined to be increased by the current load expected change amount, does not exceed the respective current load expected change amount before adjustment.

In one example, the load change rate determination module may be configured to: setting the current load expected rate of change to be greater than a previous load expected rate of change when a current health of the streaming data processing resource is not less than the previous health and the previous load expected amount of change does not exceed a first threshold; setting the current load expected rate of change to be less than the previous load expected rate of change when the current health of the streaming data processing resource is not less than the previous health and the previous load change amount exceeds the first threshold; setting an absolute value of the current load expected rate of change to be greater than an absolute value of the prior load expected rate of change when the current health of the streaming data processing resource is less than a prior health and the absolute value of the prior load expected amount of change does not exceed a second threshold; or when the current health of the streaming data processing resource is lower than the previous health and the absolute value of the expected change amount of the previous load exceeds the second threshold, setting the absolute value of the expected change rate of the current load to be smaller than the absolute value of the expected change rate of the previous load.

After determining the expected amount of load change, the expected load amount determining unit 1140 determines the current expected load amount of each streaming data processing resource based on the current amount of work done by each streaming data processing resource and the current expected amount of load change. Then, the first data allocation probability determination unit 1140 determines a first data allocation probability for each streaming data processing resource based on the current load expected amount for each streaming data processing resource.

The user characteristic data obtaining unit 1160 is configured to obtain user characteristic data of the streaming data. After the user feature data is acquired, the second data allocation probability determination unit 1170 predicts the second data allocation probability of the streaming data for each streaming data processing resource by using the streaming data processing resource recommendation model based on the user feature data of the streaming data. Then, the third data allocation probability determination unit 1180 determines a third data allocation probability based on the first data allocation probability and the second data allocation probability.

After the third data allocation probability is determined, the streaming data allocation unit is configured 1190 to allocate the streaming data to the corresponding streaming data processing resource in each of the streaming data processing resources according to the third data allocation probability.

Embodiments of a method and apparatus for distributing streaming data according to the present disclosure are described above with reference to fig. 1 through 12. The details mentioned in the above description of the method embodiments apply equally to the embodiments of the apparatus of the present disclosure.

The apparatus for distributing streaming data of the present disclosure may be implemented in hardware, or may be implemented in software, or a combination of hardware and software. The various embodiments in this specification are described in a progressive manner, with like reference to each other.

The apparatus for distributing streaming data of the present disclosure may be implemented in hardware, or may be implemented in software, or a combination of hardware and software. In the case of software implementation, as a logical means, the device is formed by reading corresponding computer program instructions in the memory into the memory for operation through the processor of the device in which the device is located. In the present disclosure, an apparatus for distributing streaming data may be implemented, for example, with a computing device.

Fig. 13 is a block diagram of a computing device for implementing a streaming data distribution method according to one embodiment of the present disclosure. As shown in fig. 13, computing device 1300 comprises processor 1310, storage 1320, memory 1330, communication interface 1340, and internal bus 1350, and processor 1310, storage 1320, memory 1330, communication interface 1340 are connected together via bus 1350. According to one embodiment, the computing device 1300 may include at least one processor 1310 that executes at least one computer-readable instruction (i.e., the elements described above as being implemented in software) stored or encoded in a computer-readable storage medium (i.e., memory 1320).

In one embodiment, computer-executable instructions are stored in the memory 1320 that, when executed, cause the at least one processor 1310 to: acquiring the current health degree, the previous health degree and the current success rate of each streaming data processing resource; for each streaming data processing resource, determining a current load expected variation of the streaming data processing resource based on a current health and a previous health of the streaming data processing resource; determining a current load expected amount of each streaming data processing resource based on a current work amount and a current load expected variation of each streaming data processing resource; determining a first data allocation probability of each streaming data processing resource based on the current load expectation of each streaming data processing resource; and allocating the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the first data allocation probability.

It should be appreciated that the computer-executable instructions stored in the memory 1320, when executed, cause the at least one processor 1310 to perform the various operations and functions described above in connection with fig. 1-12 in the various embodiments of the present disclosure.

According to one embodiment, a program product, such as a non-transitory machine-readable medium, is provided. A non-transitory machine-readable medium may have instructions (i.e., elements described above as being implemented in software) that, when executed by a machine, cause the machine to perform various operations and functions described above in connection with fig. 1-12 in various embodiments of the present disclosure.

Specifically, a system or apparatus may be provided which is provided with a readable storage medium on which software program code implementing the functions of any of the above embodiments is stored, and which causes a computer or processor of the system or apparatus to read out and execute the instructions stored in the readable storage medium.

In this case, the program code itself read from the readable medium can realize the functions of any of the above-described embodiments, and thus the machine-readable code and the readable storage medium storing the machine-readable code form part of the present invention.

Examples of the readable storage medium include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or from the cloud via a communications network.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Not all steps and elements in the above flows and system structure diagrams are necessary, and some steps or elements may be omitted according to actual needs. The execution order of the steps is not fixed, and can be determined as required. The apparatus structures described in the above embodiments may be physical structures or logical structures, that is, some units may be implemented by the same physical entity, or some units may be implemented by a plurality of physical entities, or some units may be implemented by some components in a plurality of independent devices.

The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration," and does not mean "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described technology. However, the techniques may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.

Alternative embodiments of the present disclosure are described in detail with reference to the drawings, however, the embodiments of the present disclosure are not limited to the specific details in the embodiments, and various simple modifications may be made to the technical solutions of the embodiments of the present disclosure within the technical concept of the embodiments of the present disclosure, and the simple modifications all belong to the protective scope of the embodiments of the present disclosure.

The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for distributing streaming data, comprising:

acquiring current health degree, previous health degree and current success rate of each streaming data processing resource, wherein the current health degree comprises the health degree at the current moment, the previous health degree comprises the health degree at the previous moment, and the current success rate of each streaming data processing resource comprises the streaming data rate successfully processed by the streaming data processing resource in the current time window;

for each streaming data processing resource, determining a current load expected variation of the streaming data processing resource based on a current health degree and a previous health degree of the streaming data processing resource, where the current load expected variation includes a variation of an expected throughput of the streaming data processing resource in a next time window relative to the current throughput;

determining the current load expected quantity of each streaming data processing resource based on the current work load and the current load expected variable quantity of each streaming data processing resource;

determining a first data allocation probability of each streaming data processing resource based on the current load expectation of each streaming data processing resource; and

and distributing the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the first data distribution probability.

2. The method of claim 1, wherein prior to determining the current expected amount of load for each streaming data processing resource based on the current amount of work done and the current expected amount of change in load for the respective streaming data processing resource, the method further comprises:

adjusting the current load expected change of the respective streaming data processing resource,

wherein the sum of the adjusted current load expected change amounts of the streaming data processing resources is zero, and the current load expected change amount is determined to be that the adjusted current load expected change amount of each streaming data processing resource which is increased does not exceed the respective current load expected change amount before adjustment.

3. The method of claim 1 or 2, wherein determining, for each streaming data processing resource, based on the current health and the previous health of the streaming data processing resource, the expected amount of change in the current load of the streaming data processing resource comprises:

when the current health degree of the streaming data processing resource is not lower than the previous health degree, setting the current load expected variation to be a positive value; or

When the current health degree of the streaming data processing load is lower than the previous health degree, the expected variation of the current load is set to be a negative value.

4. The method of claim 3, further comprising:

obtaining a previous expected amount of change in load and a previous expected rate of change in load for each streaming data processing resource, an

For each streaming data processing resource, determining a current amount of load change for the streaming data processing resource based on the current health and the previous health of the streaming data processing resource comprises:

for each of the streaming data processing resources,

determining a current expected rate of change of load for the streaming data processing resource based on the current health, a previous health, the previous expected amount of change in load, and a previous expected rate of change in load for the streaming data processing resource; and

based on the determined current expected rate of change in load, a current expected amount of change in load for the streaming data processing resource is determined.

5. The method of claim 4, wherein determining the current expected rate of change of load of the streaming data processing resource based on the current health, the previous expected amount of change in load, and the previous expected rate of change in load comprises:

setting the current load expected rate of change to be greater than a previous load expected rate of change when a current health of the streaming data processing resource is not less than the previous health and the previous load expected amount of change does not exceed a first threshold;

setting the current load expected rate of change to be less than the previous load expected rate of change when the current health of the streaming data processing resource is not less than the previous health and the previous load expected amount of change exceeds the first threshold;

setting an absolute value of the current load expected rate of change to be greater than an absolute value of the prior load expected rate of change when the current health of the streaming data processing resource is less than a prior health and the absolute value of the prior load expected amount of change does not exceed a second threshold; or

Setting an absolute value of the current load expected rate of change to be less than an absolute value of the prior load expected rate of change when the current health of the streaming data processing resource is less than the prior health and the absolute value of the prior load expected rate of change exceeds the second threshold.

6. The method of claim 1, further comprising:

acquiring user characteristic data of the streaming data;

predicting a second data distribution probability of the streaming data aiming at each streaming data processing resource by utilizing a streaming data processing resource recommendation model based on the user characteristic data of the streaming data;

determining a third data allocation probability based on the first data allocation probability and the second data allocation probability; and

and distributing the streaming data to corresponding streaming data processing resources in the streaming data processing resources according to the third data distribution probability.

7. An apparatus for distributing streaming data, comprising:

a processing resource characteristic obtaining unit, configured to obtain a current health degree, a previous health degree and a current success amount of each streaming data processing resource, where the current health degree includes a health degree at a current time, the previous health degree includes a health degree at a previous time, and a current success amount of each streaming data processing resource includes a streaming data amount successfully processed by the streaming data processing resource within a current time window;

a load expected change determining unit, configured to determine, for each streaming data processing resource, a current load expected change of the streaming data processing resource based on a current health and a previous health of the streaming data processing resource, where the current load expected change includes a change of an expected throughput of the streaming data processing resource in a next time window relative to the current throughput;

a load expected amount determination unit configured to determine a current load expected amount of each streaming data processing resource based on a current work load of the streaming data processing resource and a current load expected variation amount;

a first data allocation probability determination unit configured to determine a first data allocation probability for each streaming data processing resource based on a current load expected amount of the each streaming data processing resource; and

a streaming data allocation unit configured to allocate the streaming data to a corresponding streaming data processing resource of the respective streaming data processing resources according to the first data allocation probability.

8. The apparatus of claim 7, further comprising:

a load expected change amount adjustment unit configured to adjust a current load expected change amount of each streaming data processing resource before determining the current load expected amount of the each streaming data processing resource based on a current work amount of the each streaming data processing resource and the current load expected change amount,

wherein the sum of the adjusted current load expected change amounts of the streaming data processing resources is zero, and the adjusted current load expected change amount of the streaming data processing resources, which is determined to be increased by the current load expected change amount, does not exceed the current load expected change amount before the adjustment.

9. The apparatus according to claim 7 or 8, wherein the load expected change amount determination unit is configured to:

10. The apparatus of claim 9, further comprising:

a load expected change data acquisition unit configured to acquire a preceding load expected change amount and a preceding load expected change rate of each streaming data processing resource, an

The load expected change amount determination unit includes:

a load expected change rate determination module configured to determine, for each streaming data processing resource, a current load expected change rate for that streaming data processing resource based on a current health, a previous health, the previous load expected change amount, and the previous load expected change rate for that streaming data processing resource; and

a load expected change determination module configured to determine a current load expected change of the streaming data processing resource based on the determined current load expected change rate.

11. The apparatus of claim 10, wherein the load expected rate of change determination module is configured to:

setting the current load expected rate of change to be less than the previous load expected rate of change when the current health of the streaming data processing resource is not less than the previous health and the previous load change amount exceeds the first threshold;

setting an absolute value of the current load expected rate of change to be greater than an absolute value of a prior load expected rate of change when a current health of the streaming data processing resource is less than a prior health and an absolute value of the prior load expected rate of change does not exceed a second threshold; or

Setting an absolute value of the current expected rate of change of load to be less than an absolute value of a previous expected rate of change of load when a current health of the streaming data processing resource is less than a previous health and an absolute value of the previous expected amount of change of load exceeds the second threshold.

12. The apparatus of claim 7 or 8, further comprising:

a user characteristic data acquisition unit configured to acquire user characteristic data of the streaming data;

a second data allocation probability determination unit configured to predict a second data allocation probability of the streaming data for each streaming data processing resource by using a streaming data processing resource recommendation model based on user characteristic data of the streaming data;

a third data allocation probability determination unit configured to determine a third data allocation probability based on the first data allocation probability and the second data allocation probability; and

the streaming data distribution unit is configured to:

13. A computing device, comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of any of claims 1 to 6.

14. A machine-readable storage medium storing executable instructions that, when executed, cause the machine to perform the method of any of claims 1 to 6.