CN110740146A - method and device for scheduling cache nodes and computer network system - Google Patents
method and device for scheduling cache nodes and computer network system Download PDFInfo
- Publication number
- CN110740146A CN110740146A CN201810791839.XA CN201810791839A CN110740146A CN 110740146 A CN110740146 A CN 110740146A CN 201810791839 A CN201810791839 A CN 201810791839A CN 110740146 A CN110740146 A CN 110740146A
- Authority
- CN
- China
- Prior art keywords
- rate
- transmission rate
- packet loss
- transmission
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000005540 biological transmission Effects 0.000 claims abstract description 340
- 230000003247 decreasing effect Effects 0.000 claims description 2
- 238000012360 testing method Methods 0.000 description 8
- 230000002159 abnormal effect Effects 0.000 description 4
- 239000002699 waste material Substances 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method, a device and a computer network system for scheduling cache nodes, wherein the method comprises the steps of obtaining historical statistical information of the cache nodes, determining the types of the cache nodes based on a packet loss rate-data transmission rate curve in the historical statistical information, obtaining the current data transmission rate and the current packet loss rate of the cache nodes, and scheduling the data transmission rate and the transmission time period of the cache nodes with the types of the cache nodes capable of improving the transmission rate and/or the cache nodes needing to reduce the transmission rate based on the current data transmission rate and the current packet loss rate, a multi-day average packet loss rate-time curve and a multi-day average data transmission rate-time curve in the historical statistical information.
Description
Technical Field
The invention relates to the technical field of computer networks, in particular to methods and devices for scheduling cache nodes and a computer network system.
Background
With the rapid development of cloud service business, the requirement on the quality of a computer room (in which a cache node is set) is higher and higher, when a cloud service company creates a node, the cloud service company usually estimates the bandwidth amount (maximum data transmission rate) that the node can bear according to computer room evaluation data provided by a third party or by self-test, and then according to the charging setting, the cloud service company sets approximate peak values as the limit value of the node, but as the usage time of node equipment becomes longer, the bandwidth magnitude that can be borne in different time periods changes correspondingly, only a fixed bandwidth peak value is referred to be unfavorable for reasonable scheduling of resources, and problems that the node that can continue to be added cannot be completely utilized to cause bandwidth waste, and the computer room network is abnormal because the bandwidth amount of the node that cannot be added is too high often occur.
More specifically, when a cloud service company needs to newly build a machine room node or expand a node, the machine room allocates a corresponding bandwidth according to a node requirement, and then the cloud service company performs timely tests before the new node is online to roughly test whether the machine room can bear a corresponding bandwidth value or not.
After the machine room allocates the corresponding bandwidth carrying capacity according to the requirements of the cloud service company, the cloud service company determines the data transmission rate value according to the cloud service company, the node is subjected to pressure test before the node is on line, and if the bandwidth is more than x% of the peak bandwidth and the packet loss rate is less than y%, the quality of the node in the network room is considered to be normal, the node passes the test, then, the node group is added into the formal node group to schedule the bandwidth according to the bandwidth upper limit value (for example, z% of the peak bandwidth, which is less than x%), but the test result can only feed back the load capacity of the machine room at that time, when the client is actually served online, the situation that the quality of the machine room is abnormal occurs often when the client is not served according to the scheduling of the previous tested value, thereby causing the service quality to be reduced and the bandwidth bearing can not meet the requirement of the previous test to cause abnormal conditions. In addition, some nodes can actually bear bandwidth values larger than the preset upper limit value but do not bear corresponding bandwidth values which can actually bear bandwidth values, which causes bandwidth waste. In this process, because the upper limit value obtained by the test is different from the actual tolerable bandwidth value, and the time for which a certain bandwidth value can be continuously served is unknown, an abnormal condition often occurs, so that the upper limit value of the corresponding node bandwidth needs to be adjusted after manual analysis, and the processing efficiency is low and inaccurate.
In order to solve the above problems, a new technical solution needs to be proposed.
Disclosure of Invention
The method for scheduling the cache nodes comprises the following steps:
acquiring historical statistical information of cache nodes;
determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
acquiring the current data transmission rate and the current packet loss rate of a cache node;
based on the current data transmission rate and the current packet loss rate, and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information, the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate are scheduled,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rates are all less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rates are increased along with the increase of the transmission rate and the packet loss rate at the current data transmission rate is more than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rates at any transmission rate which is less than the preset transmission rate are more than or equal to the preset packet loss rate.
According to the method for scheduling the cache nodes, the step of acquiring the historical statistical information of the cache nodes comprises the following steps:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
According to the method for scheduling the cache nodes, the historical statistical information is obtained based on the filtered original collected data.
According to the method for scheduling the cache nodes, the step of scheduling the data transmission rate and the transmission time period of the cache nodes of which the types are the cache nodes capable of increasing the transmission rate and/or the cache nodes needing to reduce the transmission rate based on the current data transmission rate and the current packet loss rate and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information comprises the following steps:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate , wherein the optional transmission rate can enable the packet loss rate to be continuously less than the predetermined packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and greater than the current data transmission rate and/or the cache node needing to reduce the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and less than the current data transmission rate;
or more transmission rates and transmission time periods are selected from the optional transmission rates and the optional transmission time periods corresponding to the optional transmission rates , and the selected transmission rates and transmission time periods are used as or more transmission rates and transmission time periods for scheduling the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to decrease the transmission rates.
The device for scheduling the cache nodes comprises the following components:
the history statistical information acquisition module is used for acquiring the history statistical information of the cache nodes;
the cache node type determining module is used for determining the type of the cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
the current information acquisition module is used for acquiring the current data transmission rate and the current packet loss rate of the cache node;
a scheduling module, configured to schedule the data transmission rate and the transmission time period of the cache node of which the type is a cache node capable of increasing the transmission rate and/or a cache node requiring to decrease the transmission rate based on the current data transmission rate and the current packet loss rate, and the daily average packet loss rate-time curve and the daily average data transmission rate-time curve in the historical statistical information,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rates are all less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rates are increased along with the increase of the transmission rate and the packet loss rate at the current data transmission rate is more than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rates at any transmission rate which is less than the preset transmission rate are more than or equal to the preset packet loss rate.
According to the apparatus for scheduling cache nodes of the present invention, the historical statistical information obtaining module is further configured to:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
According to the device for scheduling the cache nodes, the historical statistical information is the statistical information obtained based on the filtered original collected data.
According to the apparatus for scheduling cache nodes of the present invention, the scheduling module is further configured to:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate , wherein the optional transmission rate can enable the packet loss rate to be continuously less than the predetermined packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and greater than the current data transmission rate and/or the cache node needing to reduce the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and less than the current data transmission rate;
or more transmission rates and transmission time periods are selected from the optional transmission rates and the optional transmission time periods corresponding to the optional transmission rates , and the selected transmission rates and transmission time periods are used as or more transmission rates and transmission time periods for scheduling the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to decrease the transmission rates.
The computer network system according to the invention comprises a source station, a cache node, a database, a load balancer or scheduler and a user client, wherein the load balancer or scheduler comprises the device for scheduling cache nodes as described above, and the database is used for storing historical statistical information of the cache node.
According to the technical scheme of the invention, the upper limit of the data transmission rate borne by different types of cache nodes can be obtained based on historical statistical information, and on the premise of meeting the data transmission quality requirement of the cache nodes, the time period for the different types of nodes to continuously bear the appointed data transmission rate is found out, so that automatic data transmission scheduling is carried out.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of the specification , illustrate embodiments of the invention and, together with a related written description , serve to explain the principles of the invention.
Fig. 1 schematically shows a flow chart of a method of scheduling cache nodes according to the present invention.
Fig. 2 schematically shows a block schematic of an arrangement for scheduling cache nodes according to the present invention.
Fig. 3 schematically shows a structural diagram of computer network systems including the apparatus for scheduling cache nodes according to the present invention.
Fig. 4 exemplarily shows a packet loss rate-data transmission rate curve corresponding to a cache node that can increase a transmission rate.
Fig. 5 exemplarily shows a packet loss rate-data transmission rate curve corresponding to a cache node whose transmission rate needs to be lowered.
Fig. 6 exemplarily shows a multi-day average packet loss rate-time curve and a multi-day average data transmission rate-time curve that may provide a basis for scheduling operations.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention.
Fig. 1 schematically shows a flow chart of a method of scheduling cache nodes according to the present invention.
As shown in fig. 1, a method for scheduling cache nodes according to the present invention includes:
step S102: acquiring historical statistical information of cache nodes;
step S104: determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
step S106: acquiring the current data transmission rate and the current packet loss rate of a cache node;
step S108: based on the current data transmission rate and the current packet loss rate, and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information, the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate are scheduled,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rates are all less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rates are increased along with the increase of the transmission rate and the packet loss rate at the current data transmission rate is more than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rates at any transmission rate which is less than the preset transmission rate are more than or equal to the preset packet loss rate.
For example, step S104 may be implemented by the following specific steps:
1. and acquiring the filtered data, and judging whether the highest packet loss rate corresponding to the bandwidth data of each x axis and the packet loss rates corresponding to other bandwidth values are obviously increased or not.
2. If there is no ascending trend (as shown in fig. 4), it indicates that the current bandwidth carrying capacity of the node does not affect the quality of the machine room, and the node is automatically set as a class a node in the time period (i.e., the above-mentioned cache node capable of increasing the transmission rate — the current upper limit is low, and an incremental scheduling node is needed).
3. If there is a significant rising trend (as shown in fig. 5), it indicates that the current bandwidth carrying capacity of the node may rise with the rise of the bandwidth carrying capacity, and then a confidence interval is given by the density degree, as shown in fig. 5, when the data transmission rate x is 10000, the probability of the packet loss rate between 3% and 4% is 15%, the probability of the packet loss rate between 2% and 3% is 25%, and the probability of the packet loss rate below 2% is 60%. And automatically setting the node as the B type in the time period (namely, the cache node needing to reduce the transmission rate, namely the upper limit value is possibly too high, and a decrement scheduling node is needed).
4. Generally, lowest limits can be set for the packet loss rate of a room (cache node) according to service categories (e.g., web pages, streaming media, etc.), for example, if the packet loss rate is greater than a% (0< a <10), then the room service capability is considered to be reduced at this time, therefore, the room with the problem (i.e., cache node) is set as the class C in this time period (i.e., the above-mentioned cache node which cannot perform data transmission, i.e., room problem, needs to be handled by the room and is not suitable for service) even if the bandwidth value is not higher than the room quality data and the packet loss rate is greater than or equal to a%).
5. The corresponding data is written to a database (e.g., database 303 in fig. 3).
Optionally, step S102 includes:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
For example, the quality data, bandwidth data and cost data of the computer room can be collected and automatically associated through the following specific steps:
1. the method comprises the steps that machine room quality data and node bandwidth data (such as data transmission rate) are collected (through a data collection module and not shown in the drawing), each node initiates ping and mtr detection through other nodes in the same province and other nodes in adjacent provinces of the CDN, machine room quality data of the node per minute are obtained, and meanwhile bandwidth data of the node per minute are collected.
2. The time-room quality data-node name classification (by a data classification module, not shown in the figure) into classes, the time-node bandwidth-node name classification into another class.
3. The only keys are time and node (through a data association module, not shown in the figure), if different types of keys are reached, then the only corresponding relationship is automatically formed between the time + node bandwidth data and the machine room quality data of the corresponding key, and the original collected data are respectively stored in the database (i.e. the database 303 in fig. 3).
Optionally, the historical statistics are statistics derived based on the filtered raw collected data.
In order to make the above historical statistical information more accurate, it is necessary to filter out data obviously having anomalies. For example, exception data filtering may be implemented by:
the above associated data obtained through automatic association is all data from the node having a bandwidth value, data are available every minute, and the data include data in which quality data (mainly, packet loss rate) of the machine room is reduced due to various reasons, as a whole, packet loss due to the machine room itself occupies only a small portion, and corresponding points are relatively sparse compared with points at which packet loss is caused by other reasons, so that filtering is easy.
Optionally, step S108 includes:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate , wherein the optional transmission rate can enable the packet loss rate to be continuously less than the predetermined packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and greater than the current data transmission rate and/or the cache node needing to reduce the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and less than the current data transmission rate;
or more transmission rates and transmission time periods are selected from the optional transmission rates and the optional transmission time periods corresponding to the optional transmission rates , and the selected transmission rates and transmission time periods are used as or more transmission rates and transmission time periods for scheduling the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to decrease the transmission rates.
For example, step S108 may include the following specific steps:
1. and (3) extracting node data, and extracting the time bandwidth, packet loss data and the cost line of each node (for example, 90% of peak data transmission rate-when the bandwidth of a node computer room is purchased, each node is well defined, when the bandwidth is higher than the cost line, the profit of a content distribution network service provider is not good, and all the bandwidth upper limit settings are lower than the cost line) according to the original data.
2. The upper bandwidth limit of the class A node (namely, the maximum data transmission rate which can be used for scheduling) is set as a cost line, and the upper bandwidth limit of the class B node is set as the maximum node bandwidth of which the probability value of a% of the packet loss is less than B% (the probability value is dynamically changed according to the average packet loss rate of the upper bandwidth limit of the node adjusted for multiple days, or the probability value is customized according to the requirement).
3. Finding the bandwidth value where the node approaches the upper limit value can continue to lose less than a% of the packet (i.e., the time period).
For example, the duration may be determined by respectively decreasing the original data and the set upper limit value of the bandwidth by 5% (a custom interval, which may be adjusted as needed) according to the upper limit value, and since the bandwidth model approximately exhibits a normal distribution, with 5% of the bandwidth as a dividing line, for example, the smallest 1 time segment of the multiple time segments with a packet loss rate less than a% (e.g., with 5% of the bandwidth interval as a step size) that meets the length of the continuous transmission time required for scheduling may be taken as the safety controllable safety duration of the bandwidth value.
For example, based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve that can be used for scheduling operation shown in fig. 6, the following conclusions can be drawn:
4. and sending the calculated bandwidth upper limit and the bandwidth upper limit safety duration time corresponding to the node to a decision scheduling system.
Through the steps, the purposes of automatically discovering the current condition of each online node computer room and performing bandwidth redundancy and quality-conforming safety scheduling are achieved.
For example, the specific scheduling procedure of the class B node may be implemented by the following specific steps:
1. node bandwidth cap data, node bandwidth-related security duration data, node real-time bandwidth and machine room data of the automated analysis system are obtained (e.g., by a decision module, corresponding to historical statistics obtaining module 201, infra).
2. And automatically judging that the current machine room packet loss is greater than a set threshold value and the bandwidth accords with a bandwidth packet loss model of the B-type node, scheduling according to a bandwidth difference value of bandwidth percentage, and defaulting the sustainable time of the bandwidth as the safety duration corresponding to the bandwidth of the node.
3. Finding out the class-A nodes in the same area of the node, judging the bandwidth value, judging the receivable bandwidth value, generating a scheduling task, and sending parameters required by scheduling to a scheduling module (for example, the scheduling module 207 in FIG. 2).
4. And after receiving the scheduling task, the scheduling module schedules according to the corresponding parameter value to generate a DNS (domain name system) issued task list, so that the node bandwidth is ensured to be as high as possible to the upper limit of the bandwidth but the quality data of the machine room is in a safety range.
Fig. 2 schematically shows a block schematic of an arrangement 200 for scheduling cache nodes according to the present invention.
As shown in fig. 2, the apparatus 200 for scheduling cache nodes according to the present invention includes:
a history statistical information obtaining module 201, configured to obtain history statistical information of the cache node;
a cache node type determining module 203, configured to determine the type of the cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
a current information obtaining module 205, configured to obtain a current data transmission rate and a current packet loss rate of the cache node;
a scheduling module 207, configured to schedule the data transmission rate and the transmission time period of the cache node of the type that is a cache node capable of increasing the transmission rate and/or a cache node requiring to decrease the transmission rate based on the current data transmission rate and the current packet loss rate, and the daily average packet loss rate-time curve and the daily average data transmission rate-time curve in the historical statistical information,
the type of the cache node comprises cache nodes which can not perform data transmission, the cache nodes which can improve the transmission rate are cache nodes of which the packet loss rates are all less than the preset packet loss rate when the transmission rate is any transmission rate which is less than the preset transmission rate of the cache nodes, the cache nodes which need to reduce the transmission rate are cache nodes of which the packet loss rates are increased along with the increase of the transmission rate and the packet loss rate at the current data transmission rate is more than or equal to the preset packet loss rate, and the cache nodes which can not perform data transmission are cache nodes of which the packet loss rates at any transmission rate which is less than the preset transmission rate are more than or equal to the preset packet loss rate.
Optionally, the historical statistical information obtaining module 201 is further configured to:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises a predetermined transmission rate.
Optionally, the historical statistics are statistics derived based on the filtered raw collected data.
Optionally, the scheduling module 207 is further configured to:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than a preset packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the preset packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each available optional transmission rate on the day and each optional transmission time period corresponding to each optional transmission rate , wherein the optional transmission rate can enable the packet loss rate to be continuously less than the predetermined packet loss rate when the cache node capable of improving the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and greater than the current data transmission rate and/or the cache node needing to reduce the transmission rate performs data transmission at the rate which is less than the predetermined transmission rate and less than the current data transmission rate;
or more transmission rates and transmission time periods are selected from the optional transmission rates and the optional transmission time periods corresponding to the optional transmission rates , and the selected transmission rates and transmission time periods are used as or more transmission rates and transmission time periods for scheduling the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to decrease the transmission rates.
Based on the method and the device, computer network systems are also provided.
Fig. 3 schematically shows an computer network system 300 comprising the apparatus 200 for scheduling cache nodes according to the present invention.
As shown in fig. 3, the exemplary computer network system 300 includes a source station 301, a cache node 302, a database 303, a load balancer or scheduler 304 (which includes the above-described apparatus 200 for scheduling cache nodes according to the present invention), and a user client 305.
A plurality of cache nodes 302 (3 in fig. 3) are used to cache resources of the source station 301 that the user 306 wishes to access through the client 305. The database 303 is used to store the above historical statistical information (i.e., the historical statistical information that needs to be acquired by the historical statistical information acquisition module 201 shown in fig. 2) about each cache node 302.
After receiving an access request for accessing resources of the source station 301, which is sent by the user 306 through the client 305, the load balancer or scheduler 304 can schedule the user request to 1 selected cache node of the 3 cache nodes 302 by using the above method for scheduling cache nodes described in conjunction with fig. 1 based on the historical statistical information of each cache node 302 acquired from the database 303, so as to perform reasonable scheduling according to the actual situation of each cache node.
That is, the overall framework of the exemplary computer network system 300 is mainly applicable to all network on-line (cache) nodes, and can find out the optimal (data transmission rate) upper limit value corresponding to each node and the schedule (time period) corresponding to the optimal data transmission rate upper limit value according to the actual on-line service condition through the relationship between the network data and the node bearers, so that the utilization rate of the nodes in the bandwidth (i.e., data transmission rate) range of the safety schedule is the highest, and the quality of the nodes (i.e., node room) can be guaranteed, thereby ensuring full automation of service, data acquisition and analysis scheduling, greatly saving labor cost, and improving accuracy.
For example, the bottom layer of the database 303 may be supported by mysql, and the data classification, the original data and the associated data are in different tables, and the table is provided with a primary key field "node", and the following fields may be contained in the table: time, data type, decision data, association data, and necessary process data, etc. And data operation is supported, and the content in the data can be synchronously modified when a new use case is changed. Can be kept for a long time, and can be used for inquiring reasons or resetting faults when needed at a later stage.
According to the technical scheme of the invention, based on the online real-time machine room carrying capacity and the corresponding machine room quality data (i.e. the historical statistical information), bandwidth upper limits (i.e. the data transmission rate) carried by different (cache) nodes (e.g. corresponding to the 3 different types) are found out through reasonable observation and a proper algorithm, and on the premise of meeting the machine room quality data, the time for the bandwidths of the different types of nodes to continuously carry on the bandwidths is found out, so that automatic bandwidth scheduling is performed.
According to the technical scheme of the invention, the method has the following advantages:
1. and dynamically changing the upper limit of the node according to the actual bearing capacity of the node and the data of the machine room along with the change of time, and providing the maximum bearing capacity under the condition of optimally ensuring the quality of the node machine room.
2. Even if the carrying capacity is too large, the carrying capacity of the maximum safe bandwidth is found, and the bandwidth resources of the nodes are utilized to the maximum (namely, the transmission rate is maximized).
3. The safety duration of the bandwidth percentage (namely, the transmission time period) is calculated, and the safety duration of the corresponding bandwidth is calculated under the condition of overload, so that the bandwidth is prevented from being reduced too much due to misjudgment, or the bandwidth is prevented from being wasted due to abandoning of the node.
4. The automatic acquisition and analysis are adopted to reduce the personnel investment to the maximum extent, and the scheduling efficiency is improved.
5. The problems that bandwidth waste is caused because the nodes which can be continuously added cannot be completely utilized and the nodes which cannot be added cause machine room network abnormity because the bandwidth is too high are solved.
The above-described aspects may be implemented individually or in various combinations, and such variations are within the scope of the present invention.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods disclosed above, functional modules/units in systems, devices may be implemented as software, firmware, hardware, and suitable combinations thereof, in a hardware implementation, the division between functional modules/units mentioned in the above description is not a division corresponding to physical components, e.g., physical components may have multiple functions, or functions or steps may be performed cooperatively by several physical components.
Finally, it should be noted that: the above examples are only for illustrating the technical solutions of the present invention, and are not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1, A method for scheduling cache nodes, comprising:
acquiring historical statistical information of cache nodes;
determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
acquiring the current data transmission rate and the current packet loss rate of the cache node;
scheduling the data transmission rate and the transmission time period of the cache node with the type of the cache node capable of increasing the transmission rate and/or the cache node needing to reduce the transmission rate based on the current data transmission rate and the current packet loss rate, and the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve in the historical statistical information,
the type further includes a cache node incapable of performing data transmission, where the cache node capable of increasing the transmission rate is a cache node whose packet loss rate is less than a predetermined packet loss rate when the transmission rate is any transmission rate less than the predetermined transmission rate of the cache node, the cache node requiring to decrease the transmission rate is a cache node whose packet loss rate increases with the increase of the transmission rate and whose packet loss rate at the current data transmission rate is greater than or equal to the predetermined packet loss rate, and the cache node incapable of performing data transmission is a cache node whose packet loss rate at any transmission rate less than the predetermined transmission rate is greater than or equal to the predetermined packet loss rate.
2. The method of scheduling cache nodes according to claim 1, wherein the step of obtaining historical statistics of cache nodes comprises:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining the historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises the predetermined transmission rate.
3. The method of scheduling cache nodes of claim 1 wherein the historical statistics are statistics based on filtered raw collected data.
4. The method according to claim 1, wherein the step of scheduling the data transmission rate and the transmission time period of the cache node of the type of the cache node with the transmission rate being increased and/or the cache node with the transmission rate being decreased based on the current data transmission rate and the current packet loss rate and the average packet loss rate-time curve for multiple days and the average data transmission rate-time curve for multiple days in the historical statistical information comprises:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than the predetermined packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the predetermined packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each optional transmission rate available on the day and each optional transmission time period corresponding to each optional transmission rate , wherein the optional transmission rate is capable of enabling the packet loss rate to be continuously smaller than the predetermined packet loss rate when the cache node capable of increasing the transmission rate performs data transmission at a rate smaller than the predetermined transmission rate and larger than the current data transmission rate and/or the cache node needing to decrease the transmission rate performs data transmission at a rate smaller than the predetermined transmission rate and smaller than the current data transmission rate;
or more transmission rates and transmission time periods are selected from the optional transmission rates and the optional transmission time periods corresponding to the optional transmission rates , and the selected transmission rates and transmission time periods are used as or more transmission rates and transmission time periods when the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to decrease the transmission rates are scheduled.
An apparatus for scheduling cache nodes of , comprising:
the history statistical information acquisition module is used for acquiring the history statistical information of the cache nodes;
the cache node type determining module is used for determining the type of a cache node based on a packet loss rate-data transmission rate curve in the historical statistical information;
the current information acquisition module is used for acquiring the current data transmission rate and the current packet loss rate of the cache node;
a scheduling module, configured to schedule a data transmission rate and a transmission time period of a cache node of a type that is a cache node capable of increasing a transmission rate and/or a cache node requiring a reduction in the transmission rate based on the current data transmission rate and the current packet loss rate, and a multi-day average packet loss rate-time curve and a multi-day average data transmission rate-time curve in the historical statistical information,
the type further includes a cache node incapable of performing data transmission, where the cache node capable of increasing the transmission rate is a cache node whose packet loss rate is less than a predetermined packet loss rate when the transmission rate is any transmission rate less than the predetermined transmission rate of the cache node, the cache node requiring to decrease the transmission rate is a cache node whose packet loss rate increases with the increase of the transmission rate and whose packet loss rate at the current data transmission rate is greater than or equal to the predetermined packet loss rate, and the cache node incapable of performing data transmission is a cache node whose packet loss rate at any transmission rate less than the predetermined transmission rate is greater than or equal to the predetermined packet loss rate.
6. The apparatus for scheduling cache nodes according to claim 5, wherein the historical statistical information obtaining module is further configured to:
automatically acquiring quality data, bandwidth data and cost data of the cache nodes;
automatically making statistics based on the quality data, bandwidth data and cost data, obtaining the historical statistical information,
wherein the quality data comprises a packet loss rate, the bandwidth data comprises a data transmission rate, and the cost data comprises the predetermined transmission rate.
7. The apparatus of claim 5, wherein the historical statistics are statistics derived based on filtered raw collected data.
8. The apparatus of scheduling a cache node of claim 5, wherein the scheduling module is further to:
determining that the current packet loss rate of the cache node capable of increasing the transmission rate is less than the predetermined packet loss rate and/or determining that the current packet loss rate of the cache node needing to decrease the transmission rate is greater than or equal to the predetermined packet loss rate;
based on the multi-day average packet loss rate-time curve and the multi-day average data transmission rate-time curve, finding out each optional transmission rate available on the day and each optional transmission time period corresponding to each optional transmission rate , wherein the optional transmission rate is capable of enabling the packet loss rate to be continuously smaller than the predetermined packet loss rate when the cache node capable of increasing the transmission rate performs data transmission at a rate smaller than the predetermined transmission rate and larger than the current data transmission rate and/or the cache node needing to decrease the transmission rate performs data transmission at a rate smaller than the predetermined transmission rate and smaller than the current data transmission rate;
or more transmission rates and transmission time periods are selected from the optional transmission rates and the optional transmission time periods corresponding to the optional transmission rates , and the selected transmission rates and transmission time periods are used as or more transmission rates and transmission time periods when the data transmission rates and transmission time periods of the cache nodes of which the types are the cache nodes capable of increasing the transmission rates and/or the cache nodes needing to decrease the transmission rates are scheduled.
A computer network system of comprising a source station, a cache node, a database, a load balancer or scheduler, and a user client, wherein the load balancer or scheduler comprises the apparatus for scheduling cache nodes of any of claims 1-4, and the database is used for storing historical statistics of the cache nodes.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810791839.XA CN110740146B (en) | 2018-07-18 | 2018-07-18 | Method and device for scheduling cache nodes and computer network system |
| PCT/CN2019/095694 WO2020015578A1 (en) | 2018-07-18 | 2019-07-12 | Method, apparatus, system, medium and device for scheduling cache node |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810791839.XA CN110740146B (en) | 2018-07-18 | 2018-07-18 | Method and device for scheduling cache nodes and computer network system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110740146A true CN110740146A (en) | 2020-01-31 |
| CN110740146B CN110740146B (en) | 2020-06-26 |
Family
ID=69163990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810791839.XA Active CN110740146B (en) | 2018-07-18 | 2018-07-18 | Method and device for scheduling cache nodes and computer network system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110740146B (en) |
| WO (1) | WO2020015578A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112751650A (en) * | 2020-12-28 | 2021-05-04 | 普联技术有限公司 | Method for determining sending rate of non-unicast data frame |
| CN114124838A (en) * | 2021-11-26 | 2022-03-01 | 广州华旻信息科技有限公司 | Data transmission method and device of big data platform and big data platform |
| CN116471273A (en) * | 2023-04-18 | 2023-07-21 | 广州智臣信息科技有限公司 | Method for realizing load balance of cross-network data exchange system |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113691390A (en) * | 2021-07-13 | 2021-11-23 | 中国电力科学研究院有限公司 | Cloud-end-coordinated edge node alarm system and method |
| CN114430374B (en) * | 2021-12-22 | 2024-09-27 | 天翼云科技有限公司 | Network resource allocation method and device and computer equipment |
| CN115208766B (en) * | 2022-07-29 | 2023-11-03 | 天翼云科技有限公司 | CDN bandwidth scheduling method and related device |
| CN117614645B (en) * | 2023-10-23 | 2024-08-23 | 深圳市德航智能技术有限公司 | Network data transmission encryption system in industrial computer communication |
| CN117834956B (en) * | 2023-12-18 | 2024-06-21 | 北京联广通网络科技有限公司 | Live video acceleration system and method for resource distribution and source return |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102917028A (en) * | 2012-09-26 | 2013-02-06 | 深圳好视网络科技有限公司 | Network video live broadcasting caching method and device |
| CN104301951A (en) * | 2014-09-28 | 2015-01-21 | 北京盈进科技有限公司 | Data package forwarding method and device |
| CN106455101A (en) * | 2016-10-31 | 2017-02-22 | 中国联合网络通信集团有限公司 | Wireless resource scheduling method and apparatus |
| WO2018001222A1 (en) * | 2016-06-27 | 2018-01-04 | 中兴通讯股份有限公司 | Data transmission cache queue allocation method and apparatus, and computer storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102130794B (en) * | 2011-02-28 | 2014-12-31 | 上海聚欣网络科技有限公司 | Method, device and equipment for measuring user uploading bandwidth |
| CN104636204B (en) * | 2014-12-04 | 2018-06-01 | 中国联合网络通信集团有限公司 | A kind of method for scheduling task and device |
| CN106412628B (en) * | 2015-07-30 | 2020-07-24 | 华为技术有限公司 | Bandwidth adjusting method and related equipment |
| CN105897582A (en) * | 2015-12-07 | 2016-08-24 | 乐视云计算有限公司 | Inter-node distance measurement method and system |
| CN105897845A (en) * | 2015-12-15 | 2016-08-24 | 乐视云计算有限公司 | CDN (Content Delivery Network) service node dispatching method and server |
-
2018
- 2018-07-18 CN CN201810791839.XA patent/CN110740146B/en active Active
-
2019
- 2019-07-12 WO PCT/CN2019/095694 patent/WO2020015578A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102917028A (en) * | 2012-09-26 | 2013-02-06 | 深圳好视网络科技有限公司 | Network video live broadcasting caching method and device |
| CN104301951A (en) * | 2014-09-28 | 2015-01-21 | 北京盈进科技有限公司 | Data package forwarding method and device |
| WO2018001222A1 (en) * | 2016-06-27 | 2018-01-04 | 中兴通讯股份有限公司 | Data transmission cache queue allocation method and apparatus, and computer storage medium |
| CN106455101A (en) * | 2016-10-31 | 2017-02-22 | 中国联合网络通信集团有限公司 | Wireless resource scheduling method and apparatus |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112751650A (en) * | 2020-12-28 | 2021-05-04 | 普联技术有限公司 | Method for determining sending rate of non-unicast data frame |
| CN112751650B (en) * | 2020-12-28 | 2023-02-14 | 普联技术有限公司 | Method for determining sending rate of non-unicast data frame |
| CN114124838A (en) * | 2021-11-26 | 2022-03-01 | 广州华旻信息科技有限公司 | Data transmission method and device of big data platform and big data platform |
| CN114124838B (en) * | 2021-11-26 | 2023-02-24 | 广州华旻信息科技有限公司 | Data transmission method and device for big data platform and big data platform management system |
| CN116471273A (en) * | 2023-04-18 | 2023-07-21 | 广州智臣信息科技有限公司 | Method for realizing load balance of cross-network data exchange system |
| CN116471273B (en) * | 2023-04-18 | 2023-10-20 | 广州智臣信息科技有限公司 | Method for realizing load balance of cross-network data exchange system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020015578A1 (en) | 2020-01-23 |
| CN110740146B (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110740146A (en) | method and device for scheduling cache nodes and computer network system | |
| CN107124445B (en) | data acquisition method and device | |
| US11604991B2 (en) | Multi-domain service assurance using real-time adaptive thresholds | |
| CN109167674B (en) | Service node scoring method, Domain Name System (DNS) scheduling method and server | |
| US20230140836A9 (en) | Anomaly detection method and device, terminal and storage medium | |
| CN107038071B (en) | Storm task flexible scheduling algorithm based on data flow prediction | |
| CN107204894A (en) | The monitoring method and device of network servicequality | |
| CN108924007B (en) | Big data acquisition and storage system and method of communication operation information | |
| Chen et al. | Learn-and-adapt stochastic dual gradients for network resource allocation | |
| CN104580018A (en) | Bandwidth adjustment method and device in software-defined network | |
| CN103561428A (en) | Method and system for elastically distributing nodes in short message gateway cluster system | |
| EP3981111A1 (en) | Allocating cloud resources in accordance with predicted deployment growth | |
| CN104320353A (en) | User address managing method and device | |
| CN102026228B (en) | Statistical method and equipment for communication network performance data | |
| CN117221088A (en) | Computer network intensity detection system and device | |
| KR101842996B1 (en) | Data processing apparatus of scada system | |
| CN103607731A (en) | Method and device for processing measurement reports | |
| CN113114480B (en) | Data reporting method and related equipment | |
| WO2024066720A1 (en) | Indicator threshold determination method and apparatus, storage medium, and electronic apparatus | |
| CN104125115B (en) | A kind of log information transfer approach and device | |
| CN104714875A (en) | Distributed automatic collecting method | |
| WO2022001480A1 (en) | Popular application identification method, network system, network device and storage medium | |
| CN105429852B (en) | Adaptive message handling system suitable for large capacity packet delivery system | |
| CN109936478A (en) | A kind of customer service falls the early warning analysis method and system before list | |
| CN115460124A (en) | Method, device, equipment and storage medium for cross-machine room transmission link optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40019467 Country of ref document: HK |