CN110740146B - Method and device for scheduling cache nodes and computer network system - Google Patents
Method and device for scheduling cache nodes and computer network system Download PDFInfo
- Publication number
- CN110740146B CN110740146B CN201810791839.XA CN201810791839A CN110740146B CN 110740146 B CN110740146 B CN 110740146B CN 201810791839 A CN201810791839 A CN 201810791839A CN 110740146 B CN110740146 B CN 110740146B
- Authority
- CN
- China
- Prior art keywords
- rate
- transmission rate
- packet loss
- transmission
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
Abstract
The invention discloses a method and a device for scheduling cache nodes and a computer network system. The disclosed method comprises: acquiring historical statistical information of cache nodes; determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information; acquiring the current data transmission rate and the current packet loss rate of a cache node; and scheduling the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate based on the current data transmission rate and the current packet loss rate, and a multi-day average packet loss rate-time curve and a multi-day average data transmission rate-time curve in the historical statistical information. The disclosed scheme can obtain the upper limit of the data transmission rate borne by different types of cache nodes based on historical statistical information, find out the time period in which the different types of nodes can continuously bear the appointed data transmission rate, and perform automatic data transmission scheduling.
Description
Technical Field
The present invention relates to the field of computer network technologies, and in particular, to a method and an apparatus for scheduling cache nodes, and a computer network system.
Background
With the rapid development of cloud service business, the requirement on the quality of a computer room (in which a cache node is set) is higher and higher, when a node is newly built by a cloud service company, the bandwidth amount (maximum data transmission rate) which can be borne by the node is usually estimated according to computer room evaluation data provided by a third party or self-test, then according to charging setting, the cloud service company sets an approximate peak value as a limit value of the node, but as the service time of node equipment becomes longer, the bandwidth magnitude which can be borne in different time periods changes correspondingly, only a fixed bandwidth peak value is referred to be unfavorable for reasonable scheduling of resources, and problems that the node which can continue to be added cannot be completely utilized to cause bandwidth waste, and the computer room network is abnormal because the bandwidth amount of the node which cannot be added is too high often occur.
More specifically, when a cloud service company needs to newly create a machine room node or expand a node, the machine room allocates a corresponding bandwidth according to a node requirement, and then the cloud service company performs some timely tests before the new node is on line, so as to roughly test whether the machine room can bear a corresponding bandwidth value. If the test result meets the originally set value, the cloud service company purchases the corresponding node, and then the cloud service company schedules the bandwidth of the node according to a fixed value, but the test value can only represent that the machine room at the corresponding time point meets the bandwidth bearing capacity under the quality condition.
After the machine room allocates the corresponding bandwidth carrying capacity according to the requirements of the cloud service company, the cloud service company determines the data transmission rate value according to the cloud service company, the node is subjected to pressure test before the node is on line, and if the bandwidth is more than x% of the peak bandwidth and the packet loss rate is less than y%, the quality of the node in the network room is considered to be normal, the node passes the test, then, the node group is added into the formal node group to schedule the bandwidth according to the bandwidth upper limit value (for example, z% of the peak bandwidth, which is less than x%), but the test result can only feed back the load capacity of the machine room at that time, when the client is actually served online, the situation that the quality of the machine room is abnormal occurs often when the client is not served according to the scheduling of the previous tested value, thereby causing the service quality to be reduced and the bandwidth bearing can not meet the requirement of the previous test to cause abnormal conditions. In addition, some nodes can actually bear bandwidth values larger than the preset upper limit value but do not bear corresponding bandwidth values which can actually bear bandwidth values, which causes bandwidth waste. In this process, because the upper limit value obtained by the test is different from the actual tolerable bandwidth value, and the time for which a certain bandwidth value can be continuously served is unknown, an abnormal condition often occurs, so that the upper limit value of the corresponding node bandwidth needs to be adjusted after manual analysis, and the processing efficiency is low and inaccurate.
In order to solve the above problems, a new technical solution needs to be proposed.
Disclosure of Invention
The method for scheduling the cache nodes comprises the following steps:
acquiring historical statistical information of cache nodes;
determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
acquiring the current data transmission rate and the current packet loss rate of a cache node;
based on the current data transmission rate and the current packet loss rate, and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information, the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate are scheduled,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rate is less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rate is increased along with the increase of the transmission rate and of which the packet loss rate at the current data transmission rate is greater than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rate at any transmission rate which is less than the preset transmission rate is greater than or equal to the preset packet loss rate.
According to the method for scheduling the cache nodes, the step of acquiring the historical statistical information of the cache nodes comprises the following steps:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
According to the method for scheduling the cache nodes, the historical statistical information is obtained based on the filtered original collected data.
According to the method for scheduling the cache nodes, the step of scheduling the data transmission rate and the transmission time period of the cache nodes of which the types are the cache nodes capable of increasing the transmission rate and/or the cache nodes needing to reduce the transmission rate based on the current data transmission rate and the current packet loss rate and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information comprises the following steps:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate, wherein the packet loss rate of each optional transmission rate is continuously less than the preset packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the preset transmission rate and greater than the current data transmission rate and/or the cache node required to reduce the transmission rate performs data transmission at the rate which is less than the preset transmission rate and less than the current data transmission rate;
and selecting one or more transmission rates and transmission time periods from the optional transmission rates and the optional transmission time periods in one-to-one correspondence with the optional transmission rates, wherein the selected transmission rates and transmission time periods are used for scheduling the data transmission rates and the transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to reduce the transmission rates.
The device for scheduling the cache nodes comprises the following components:
the history statistical information acquisition module is used for acquiring the history statistical information of the cache nodes;
the cache node type determining module is used for determining the type of the cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
the current information acquisition module is used for acquiring the current data transmission rate and the current packet loss rate of the cache node;
a scheduling module, configured to schedule the data transmission rate and the transmission time period of the cache node of which the type is a cache node capable of increasing the transmission rate and/or a cache node requiring to decrease the transmission rate based on the current data transmission rate and the current packet loss rate, and the daily average packet loss rate-time curve and the daily average data transmission rate-time curve in the historical statistical information,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rate is less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rate is increased along with the increase of the transmission rate and of which the packet loss rate at the current data transmission rate is greater than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rate at any transmission rate which is less than the preset transmission rate is greater than or equal to the preset packet loss rate.
According to the apparatus for scheduling cache nodes of the present invention, the historical statistical information obtaining module is further configured to:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
According to the device for scheduling the cache nodes, the historical statistical information is the statistical information obtained based on the filtered original collected data.
According to the apparatus for scheduling cache nodes of the present invention, the scheduling module is further configured to:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate, wherein the packet loss rate of each optional transmission rate is continuously less than the preset packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the preset transmission rate and greater than the current data transmission rate and/or the cache node required to reduce the transmission rate performs data transmission at the rate which is less than the preset transmission rate and less than the current data transmission rate;
and selecting one or more transmission rates and transmission time periods from the optional transmission rates and the optional transmission time periods in one-to-one correspondence with the optional transmission rates, wherein the selected transmission rates and transmission time periods are used for scheduling the data transmission rates and the transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to reduce the transmission rates.
The computer network system according to the invention comprises a source station, a cache node, a database, a load balancer or scheduler and a user client, wherein the load balancer or scheduler comprises the device for scheduling cache nodes as described above, and the database is used for storing historical statistical information of the cache node.
According to the technical scheme of the invention, the upper limit of the data transmission rate borne by different types of cache nodes can be obtained based on historical statistical information, and on the premise of meeting the data transmission quality requirement of the cache nodes, the time period for the different types of nodes to continuously bear the appointed data transmission rate is found out, so that automatic data transmission scheduling is carried out.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention. In the drawings, like reference numerals are used to indicate like elements. The drawings in the following description are directed to some, but not all embodiments of the invention. For a person skilled in the art, other figures can be derived from these figures without inventive effort.
Fig. 1 schematically shows a flow chart of a method of scheduling cache nodes according to the present invention.
Fig. 2 schematically shows a block schematic of an arrangement for scheduling cache nodes according to the present invention.
Fig. 3 is a schematic structural diagram of a computer network system including an apparatus for scheduling cache nodes according to the present invention.
Fig. 4 exemplarily shows a packet loss rate-data transmission rate curve corresponding to a cache node that can increase a transmission rate.
Fig. 5 exemplarily shows a packet loss rate-data transmission rate curve corresponding to a cache node whose transmission rate needs to be lowered.
Fig. 6 exemplarily shows a multi-day average packet loss rate-time curve and a multi-day average data transmission rate-time curve that may provide a basis for scheduling operations.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
Fig. 1 schematically shows a flow chart of a method of scheduling cache nodes according to the present invention.
As shown in fig. 1, a method for scheduling cache nodes according to the present invention includes:
step S102: acquiring historical statistical information of cache nodes;
step S104: determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
step S106: acquiring the current data transmission rate and the current packet loss rate of a cache node;
step S108: based on the current data transmission rate and the current packet loss rate, and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information, the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate are scheduled,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rate is less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rate is increased along with the increase of the transmission rate and of which the packet loss rate at the current data transmission rate is greater than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rate at any transmission rate which is less than the preset transmission rate is greater than or equal to the preset packet loss rate.
For example, step S104 may be implemented by the following specific steps:
1. and acquiring the filtered data, and judging whether the highest packet loss rate corresponding to the bandwidth data of each x axis and the packet loss rates corresponding to other bandwidth values are obviously increased or not.
2. If there is no ascending trend (as shown in fig. 4), it indicates that the current bandwidth carrying capacity of the node does not affect the quality of the machine room, and the node is automatically set as a class a node in the time period (i.e., the above-mentioned cache node capable of increasing the transmission rate — the current upper limit is low, and an incremental scheduling node is needed).
3. If there is a significant rising trend (as shown in fig. 5), it indicates that the current bandwidth carrying capacity of the node may rise with the rise of the bandwidth carrying capacity, and then a confidence interval is given by the density degree, as shown in fig. 5, when the data transmission rate x is 10000, the probability of the packet loss rate between 3% and 4% is 15%, the probability of the packet loss rate between 2% and 3% is 25%, and the probability of the packet loss rate below 2% is 60%. And automatically setting the node as the B type in the time period (namely, the cache node needing to reduce the transmission rate, namely the upper limit value is possibly too high, and a decrement scheduling node is needed).
4. Generally, a minimum limit may be set on the packet loss rate of the room (cache node) according to the service type (e.g., web page, streaming media, etc.), for example, if the packet loss rate is greater than a% (0< a <10), the room service capability is considered to be reduced at this time. Therefore, if a problem exists in the machine room (i.e., a cache node), even if the bandwidth value is not high, the machine room quality data is dense and the packet loss rate in the dense area is greater than or equal to a%, the node is set to be class C in the time period (i.e., the above-mentioned cache node which cannot perform data transmission — the machine room problem needs to be handled by the machine room and is not suitable for service).
5. The corresponding data is written to a database (e.g., database 303 in fig. 3).
Optionally, step S102 includes:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
For example, the quality data, bandwidth data and cost data of the computer room can be collected and automatically associated through the following specific steps:
1. the method comprises the steps that machine room quality data and node bandwidth data (such as data transmission rate) are collected (through a data collection module and not shown in the drawing), each node initiates ping and mtr detection through other nodes in the same province and other nodes in adjacent provinces of the CDN, machine room quality data of the node per minute are obtained, and meanwhile bandwidth data of the node per minute are collected.
2. The time-machine room quality data-node names are classified into one class and the time-node bandwidth-node names are classified into another class (by a data classification module, not shown in the drawing).
3. The time and the node are used as unique keys (through a data association module, not shown in the drawing), if the keys of different types are consistent, the time + node bandwidth data and the machine room quality data of the corresponding key form a unique corresponding relation, and the original collected data are respectively stored into a database (namely, the database 303 in fig. 3).
Optionally, the historical statistics are statistics derived based on the filtered raw collected data.
In order to make the above historical statistical information more accurate, it is necessary to filter out data obviously having anomalies. For example, exception data filtering may be implemented by:
the above association data obtained by automatic association is all data of the node since the node has a bandwidth value, and each minute has one data, and the data includes data in which quality data (mainly packet loss rate) of the computer room is reduced due to various reasons. On the whole, the packet loss ratio caused by the machine room itself only occupies a small part, and the corresponding points are sparse relative to the points of packet loss caused by other reasons, so that the points are easy to filter. For example, outliers can be filtered out by the 3 σ algorithm of normal distribution.
Optionally, step S108 includes:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate, wherein the packet loss rate of each optional transmission rate is continuously less than the preset packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the preset transmission rate and greater than the current data transmission rate and/or the cache node required to reduce the transmission rate performs data transmission at the rate which is less than the preset transmission rate and less than the current data transmission rate;
and selecting one or more transmission rates and transmission time periods from the optional transmission rates and the optional transmission time periods in one-to-one correspondence with the optional transmission rates, wherein the selected transmission rates and transmission time periods are used for scheduling the data transmission rates and the transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to reduce the transmission rates.
For example, step S108 may include the following specific steps:
1. and (3) extracting node data, and extracting the time bandwidth, packet loss data and the cost line of each node (for example, 90% of peak data transmission rate-when the bandwidth of a node computer room is purchased, each node is well defined, when the bandwidth is higher than the cost line, the profit of a content distribution network service provider is not good, and all the bandwidth upper limit settings are lower than the cost line) according to the original data.
2. The upper bandwidth limit of the class A node (namely, the maximum data transmission rate which can be used for scheduling) is set as a cost line, and the upper bandwidth limit of the class B node is set as the maximum node bandwidth of which the probability value of a% of the packet loss is less than B% (the probability value is dynamically changed according to the average packet loss rate of the upper bandwidth limit of the node adjusted for multiple days, or the probability value is customized according to the requirement).
3. Finding the bandwidth value where the node approaches the upper limit value can continue to lose less than a% of the packet (i.e., the time period).
For example, the duration may be determined by respectively decreasing the original data and the set upper limit value of the bandwidth by 5% (a custom interval, which may be adjusted as needed) according to the upper limit value, and since the bandwidth model approximately exhibits a normal distribution, with 5% of the bandwidth as a dividing line, for example, the smallest 1 time segment of the multiple time segments with a packet loss rate less than a% (e.g., with 5% of the bandwidth interval as a step size) that meets the length of the continuous transmission time required for scheduling may be taken as the safety controllable safety duration of the bandwidth value.
For example, based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve that can be used for scheduling operation shown in fig. 6, the following conclusions can be drawn:
4. and sending the calculated bandwidth upper limit and the bandwidth upper limit safety duration time corresponding to the node to a decision scheduling system.
Through the steps, the purposes of automatically discovering the current condition of each online node computer room and performing bandwidth redundancy and quality-conforming safety scheduling are achieved.
For example, the specific scheduling procedure of the class B node may be implemented by the following specific steps:
1. node bandwidth cap data, node bandwidth-related security duration data, node real-time bandwidth and machine room data of the automated analysis system are obtained (e.g., by a decision module, corresponding to historical statistics obtaining module 201, infra).
2. And automatically judging that the current machine room packet loss is greater than a set threshold value and the bandwidth accords with a bandwidth packet loss model of the B-type node, scheduling according to a bandwidth difference value of bandwidth percentage, and defaulting the sustainable time of the bandwidth as the safety duration corresponding to the bandwidth of the node.
3. Finding out the class-A nodes in the same area of the node, judging the bandwidth value, judging the receivable bandwidth value, generating a scheduling task, and sending parameters required by scheduling to a scheduling module (for example, the scheduling module 207 in FIG. 2).
4. And after receiving the scheduling task, the scheduling module schedules according to the corresponding parameter value to generate a DNS (domain name system) issued task list, so that the node bandwidth is ensured to be as high as possible to the upper limit of the bandwidth but the quality data of the machine room is in a safety range.
Fig. 2 schematically shows a block schematic of an arrangement 200 for scheduling cache nodes according to the present invention.
As shown in fig. 2, the apparatus 200 for scheduling cache nodes according to the present invention includes:
a history statistical information obtaining module 201, configured to obtain history statistical information of the cache node;
a cache node type determining module 203, configured to determine the type of the cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
a current information obtaining module 205, configured to obtain a current data transmission rate and a current packet loss rate of the cache node;
a scheduling module 207, configured to schedule the data transmission rate and the transmission time period of the cache node of the type that is a cache node capable of increasing the transmission rate and/or a cache node requiring to decrease the transmission rate based on the current data transmission rate and the current packet loss rate, and the daily average packet loss rate-time curve and the daily average data transmission rate-time curve in the historical statistical information,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rate is less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rate is increased along with the increase of the transmission rate and of which the packet loss rate at the current data transmission rate is greater than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rate at any transmission rate which is less than the preset transmission rate is greater than or equal to the preset packet loss rate.
Optionally, the historical statistical information obtaining module 201 is further configured to:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
Optionally, the historical statistics are statistics derived based on the filtered raw collected data.
Optionally, the scheduling module 207 is further configured to:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate, wherein the packet loss rate of each optional transmission rate is continuously less than the preset packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the preset transmission rate and greater than the current data transmission rate and/or the cache node required to reduce the transmission rate performs data transmission at the rate which is less than the preset transmission rate and less than the current data transmission rate;
and selecting one or more transmission rates and transmission time periods from the optional transmission rates and the optional transmission time periods in one-to-one correspondence with the optional transmission rates, wherein the selected transmission rates and transmission time periods are used for scheduling the data transmission rates and the transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to reduce the transmission rates.
Based on the method and the device, the invention also provides a computer network system.
Fig. 3 schematically shows a structural diagram of a computer network system 300 comprising an apparatus 200 for scheduling cache nodes according to the present invention.
As shown in fig. 3, the exemplary computer network system 300 includes a source station 301, a cache node 302, a database 303, a load balancer or scheduler 304 (which includes the above-described apparatus 200 for scheduling cache nodes according to the present invention), and a user client 305.
A plurality of cache nodes 302 (3 in fig. 3) are used to cache resources of the source station 301 that the user 306 wishes to access through the client 305. The database 303 is used to store the above historical statistical information (i.e., the historical statistical information that needs to be acquired by the historical statistical information acquisition module 201 shown in fig. 2) about each cache node 302.
After receiving an access request for accessing resources of the source station 301, which is sent by the user 306 through the client 305, the load balancer or scheduler 304 can schedule the user request to 1 selected cache node of the 3 cache nodes 302 by using the above method for scheduling cache nodes described in conjunction with fig. 1 based on the historical statistical information of each cache node 302 acquired from the database 303, so as to perform reasonable scheduling according to the actual situation of each cache node.
That is, the overall framework of the exemplary computer network system 300 is mainly applicable to all network on-line (cache) nodes, and can find out the optimal (data transmission rate) upper limit value corresponding to each node and the schedule (time period) corresponding to the optimal data transmission rate upper limit value according to the actual on-line service condition through the relationship between the network data and the node bearers, so that the utilization rate of the nodes in the bandwidth (i.e., data transmission rate) range of the safety schedule is the highest, and the quality of the nodes (i.e., node room) can be guaranteed, thereby ensuring full automation of service, data acquisition and analysis scheduling, greatly saving labor cost, and improving accuracy.
For example, the bottom layer of the database 303 may be supported by mysql, and the data classification, the original data and the associated data are in different tables, and the table is provided with a primary key field "node", and the following fields may be contained in the table: time, data type, decision data, association data, and necessary process data, etc. And data operation is supported, and the content in the data can be synchronously modified when a new use case is changed. Can be kept for a long time, and can be used for inquiring reasons or resetting faults when needed at a later stage.
According to the technical scheme of the invention, based on the online real-time machine room carrying capacity and the corresponding machine room quality data (i.e. the historical statistical information), bandwidth upper limits (i.e. the data transmission rate) carried by different (cache) nodes (e.g. corresponding to the 3 different types) are found out through reasonable observation and a proper algorithm, and on the premise of meeting the machine room quality data, the time for the bandwidths of the different types of nodes to continuously carry on the bandwidths is found out, so that automatic bandwidth scheduling is performed.
According to the technical scheme of the invention, the method has the following advantages:
1. and dynamically changing the upper limit of the node according to the actual bearing capacity of the node and the data of the machine room along with the change of time, and providing the maximum bearing capacity under the condition of optimally ensuring the quality of the node machine room.
2. Even if the carrying capacity is too large, the carrying capacity of the maximum safe bandwidth is found, and the bandwidth resources of the nodes are utilized to the maximum (namely, the transmission rate is maximized).
3. The safety duration of the bandwidth percentage (namely, the transmission time period) is calculated, and the safety duration of the corresponding bandwidth is calculated under the condition of overload, so that the bandwidth is prevented from being reduced too much due to misjudgment, or the bandwidth is prevented from being wasted due to abandoning of the node.
4. The automatic acquisition and analysis are adopted to reduce the personnel investment to the maximum extent, and the scheduling efficiency is improved.
5. The problems that bandwidth waste is caused because the nodes which can be continuously added cannot be completely utilized and the nodes which cannot be added cause machine room network abnormity because the bandwidth is too high are solved.
The above-described aspects may be implemented individually or in various combinations, and such variations are within the scope of the present invention.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Finally, it should be noted that: the above examples are only for illustrating the technical solutions of the present invention, and are not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. A method for scheduling cache nodes, comprising:
acquiring historical statistical information of cache nodes;
determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
acquiring the current data transmission rate and the current packet loss rate of the cache node;
scheduling the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate based on the current data transmission rate and the current packet loss rate, and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information,
the type further includes a cache node incapable of performing data transmission, the cache node capable of increasing the transmission rate is a cache node of which the packet loss rate is less than a predetermined packet loss rate when the transmission rate is any one of the transmission rates less than the predetermined transmission rate of the cache node, the cache node requiring to decrease the transmission rate is a cache node of which the packet loss rate increases with the increase of the transmission rate and of which the packet loss rate at the current data transmission rate is greater than or equal to the predetermined packet loss rate, and the cache node incapable of performing data transmission is a cache node of which the packet loss rate at any one of the transmission rates less than the predetermined transmission rate is greater than or equal to the predetermined packet loss rate.
2. The method of scheduling cache nodes according to claim 1, wherein the step of obtaining historical statistics of cache nodes comprises:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining the historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises the predetermined transmission rate.
3. The method of scheduling cache nodes of claim 1 wherein the historical statistics are statistics based on filtered raw collected data.
4. The method according to claim 1, wherein the step of scheduling the data transmission rate and the transmission time period of the cache node of the type of the cache node with the transmission rate being increased and/or the cache node with the transmission rate being decreased based on the current data transmission rate and the current packet loss rate and the average packet loss rate-time curve for multiple days and the average data transmission rate-time curve for multiple days in the historical statistical information comprises:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than the predetermined packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the predetermined packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out that when the cache node capable of increasing the transmission rate performs data transmission at a rate which is less than the predetermined transmission rate and greater than the current data transmission rate and/or the cache node needing to decrease the transmission rate performs data transmission at a rate which is less than the predetermined transmission rate and less than the current data transmission rate, each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate, wherein the packet loss rate can be continuously less than the predetermined packet loss rate;
and selecting one or more transmission rates and transmission time periods from the optional transmission rates and the optional transmission time periods in one-to-one correspondence with the optional transmission rates, wherein the selected transmission rates and transmission time periods are used for scheduling the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to reduce the transmission rates.
5. An apparatus for scheduling cache nodes, comprising:
the history statistical information acquisition module is used for acquiring the history statistical information of the cache nodes;
the cache node type determining module is used for determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
the current information acquisition module is used for acquiring the current data transmission rate and the current packet loss rate of the cache node;
a scheduling module, configured to schedule a data transmission rate and a transmission time period of a cache node of a type that is a cache node capable of increasing a transmission rate and/or a cache node requiring a reduction in the transmission rate based on the current data transmission rate and the current packet loss rate, and a multi-day average packet loss rate-time curve and a multi-day average data transmission rate-time curve in the historical statistical information,
the type further includes a cache node incapable of performing data transmission, the cache node capable of increasing the transmission rate is a cache node of which the packet loss rate is less than a predetermined packet loss rate when the transmission rate is any one of the transmission rates less than the predetermined transmission rate of the cache node, the cache node requiring to decrease the transmission rate is a cache node of which the packet loss rate increases with the increase of the transmission rate and of which the packet loss rate at the current data transmission rate is greater than or equal to the predetermined packet loss rate, and the cache node incapable of performing data transmission is a cache node of which the packet loss rate at any one of the transmission rates less than the predetermined transmission rate is greater than or equal to the predetermined packet loss rate.
6. The apparatus for scheduling cache nodes according to claim 5, wherein the historical statistical information obtaining module is further configured to:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining the historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises the predetermined transmission rate.
7. The apparatus of claim 5, wherein the historical statistics are statistics derived based on filtered raw collected data.
8. The apparatus of scheduling a cache node of claim 5, wherein the scheduling module is further to:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than the predetermined packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the predetermined packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out that when the cache node capable of increasing the transmission rate performs data transmission at a rate which is less than the predetermined transmission rate and greater than the current data transmission rate and/or the cache node needing to decrease the transmission rate performs data transmission at a rate which is less than the predetermined transmission rate and less than the current data transmission rate, each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate, wherein the packet loss rate can be continuously less than the predetermined packet loss rate;
and selecting one or more transmission rates and transmission time periods from the optional transmission rates and the optional transmission time periods in one-to-one correspondence with the optional transmission rates, wherein the selected transmission rates and transmission time periods are used for scheduling the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to reduce the transmission rates.
9. A computer network system comprising a source station, a cache node, a database, a load balancer and a user client, or a source station, a cache node, a database, a scheduler and a user client, characterized in that the load balancer or the scheduler comprises an apparatus for scheduling cache nodes according to any of claims 1-4, the database being used for storing historical statistics of the cache nodes.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810791839.XA CN110740146B (en) | 2018-07-18 | 2018-07-18 | Method and device for scheduling cache nodes and computer network system |
| PCT/CN2019/095694 WO2020015578A1 (en) | 2018-07-18 | 2019-07-12 | Method, apparatus, system, medium and device for scheduling cache node |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810791839.XA CN110740146B (en) | 2018-07-18 | 2018-07-18 | Method and device for scheduling cache nodes and computer network system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110740146A CN110740146A (en) | 2020-01-31 |
| CN110740146B true CN110740146B (en) | 2020-06-26 |
Family
ID=69163990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810791839.XA Active CN110740146B (en) | 2018-07-18 | 2018-07-18 | Method and device for scheduling cache nodes and computer network system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110740146B (en) |
| WO (1) | WO2020015578A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112751650B (en) * | 2020-12-28 | 2023-02-14 | 普联技术有限公司 | Method for determining sending rate of non-unicast data frame |
| CN113691390A (en) * | 2021-07-13 | 2021-11-23 | 中国电力科学研究院有限公司 | Cloud-end-coordinated edge node alarm system and method |
| CN114124838B (en) * | 2021-11-26 | 2023-02-24 | 广州华旻信息科技有限公司 | Data transmission method and device for big data platform and big data platform management system |
| CN115208766B (en) * | 2022-07-29 | 2023-11-03 | 天翼云科技有限公司 | CDN bandwidth scheduling method and related device |
| CN116471273B (en) * | 2023-04-18 | 2023-10-20 | 广州智臣信息科技有限公司 | Method for realizing load balance of cross-network data exchange system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102917028A (en) * | 2012-09-26 | 2013-02-06 | 深圳好视网络科技有限公司 | Network video live broadcasting caching method and device |
| CN104301951A (en) * | 2014-09-28 | 2015-01-21 | 北京盈进科技有限公司 | Data package forwarding method and device |
| CN106455101A (en) * | 2016-10-31 | 2017-02-22 | 中国联合网络通信集团有限公司 | Wireless resource scheduling method and apparatus |
| WO2018001222A1 (en) * | 2016-06-27 | 2018-01-04 | 中兴通讯股份有限公司 | Data transmission cache queue allocation method and apparatus, and computer storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102130794B (en) * | 2011-02-28 | 2014-12-31 | 上海聚欣网络科技有限公司 | Method, device and equipment for measuring user uploading bandwidth |
| CN104636204B (en) * | 2014-12-04 | 2018-06-01 | 中国联合网络通信集团有限公司 | A kind of method for scheduling task and device |
| CN106412628B (en) * | 2015-07-30 | 2020-07-24 | 华为技术有限公司 | Bandwidth adjusting method and related equipment |
| CN105897582A (en) * | 2015-12-07 | 2016-08-24 | 乐视云计算有限公司 | Inter-node distance measurement method and system |
| CN105897845A (en) * | 2015-12-15 | 2016-08-24 | 乐视云计算有限公司 | CDN (Content Delivery Network) service node dispatching method and server |
-
2018
- 2018-07-18 CN CN201810791839.XA patent/CN110740146B/en active Active
-
2019
- 2019-07-12 WO PCT/CN2019/095694 patent/WO2020015578A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102917028A (en) * | 2012-09-26 | 2013-02-06 | 深圳好视网络科技有限公司 | Network video live broadcasting caching method and device |
| CN104301951A (en) * | 2014-09-28 | 2015-01-21 | 北京盈进科技有限公司 | Data package forwarding method and device |
| WO2018001222A1 (en) * | 2016-06-27 | 2018-01-04 | 中兴通讯股份有限公司 | Data transmission cache queue allocation method and apparatus, and computer storage medium |
| CN106455101A (en) * | 2016-10-31 | 2017-02-22 | 中国联合网络通信集团有限公司 | Wireless resource scheduling method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110740146A (en) | 2020-01-31 |
| WO2020015578A1 (en) | 2020-01-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110740146B (en) | Method and device for scheduling cache nodes and computer network system | |
| CN107124445B (en) | data acquisition method and device | |
| US20170331705A1 (en) | Resource Scaling Method on Cloud Platform and Cloud Platform | |
| CN102594675B (en) | Traffic control system and method | |
| US11604991B2 (en) | Multi-domain service assurance using real-time adaptive thresholds | |
| CN108737132B (en) | Alarm information processing method and device | |
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| CN110581801B (en) | Network adjustment method and device for content distribution network | |
| CN108924007B (en) | Big data acquisition and storage system and method of communication operation information | |
| CN111694517A (en) | Distributed data migration method and system and electronic equipment | |
| CN111291252B (en) | Method and device for adjusting query rate per second, electronic equipment and storage medium | |
| CN113391973A (en) | Internet of things cloud container log collection method and device | |
| CN103607731A (en) | Method and device for processing measurement reports | |
| CN111262783B (en) | Dynamic routing method and device | |
| CN109450672B (en) | Method and device for identifying bandwidth demand burst | |
| CN101227330B (en) | Method and system for history performance data collection | |
| CN112887224B (en) | Traffic scheduling processing method and device, electronic equipment and storage medium | |
| WO2022001480A1 (en) | Popular application identification method, network system, network device and storage medium | |
| CN105429852B (en) | Adaptive message handling system suitable for large capacity packet delivery system | |
| CN112751926B (en) | Management method, system and related device for working nodes in cluster | |
| CN102547789A (en) | Early warning method, device and system for quality of peer-to-peer service | |
| US20150032875A1 (en) | Adaptive polling of information from a device | |
| CN107493308B (en) | Method and device for sending message and distributed equipment cluster system | |
| CN113114480B (en) | Data reporting method and related equipment | |
| CN111343244B (en) | Cloud distribution network scheduling method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40019467 Country of ref document: HK |