CN109347763B - Data scheduling method, device and system based on data queue length - Google Patents
Data scheduling method, device and system based on data queue length Download PDFInfo
- Publication number
- CN109347763B CN109347763B CN201811054783.6A CN201811054783A CN109347763B CN 109347763 B CN109347763 B CN 109347763B CN 201811054783 A CN201811054783 A CN 201811054783A CN 109347763 B CN109347763 B CN 109347763B
- Authority
- CN
- China
- Prior art keywords
- data
- queue
- edge node
- virtual
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
Abstract
The embodiment of the invention provides a data scheduling method, a system and a device based on the length of a data queue, wherein the method comprises the following steps: determining each destination node in the network according to the network topology structure, wherein the destination node comprises an edge node and a terminal; determining a virtual data value of a data queue corresponding to each target node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0; and setting the virtual data value of the data queue of each destination node so that each destination node takes the sum of the corresponding virtual data value and the actual storage data amount as the total data amount of the data queue, and performing data processing based on the total data amount. Because the virtual data value of each destination node does not occupy the storage space, and each network node takes the sum of the virtual data value and the actual storage data amount as the total data amount of the data queue, the data transmission to the upper layer of the network is started when the data amount of the actual arrival data does not meet the threshold condition, and the network working efficiency is improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data scheduling method, device, and system based on a data queue length.
Background
Due to the limited network resources, such as the computational power of the network nodes, the link capacity of the network, etc., there may be a pile of unprocessed or transmitted data in the network, thereby causing network congestion. For such queuing networks, a scheduling method is generally adopted, in which the length of a data queue of each network node is taken as a metric, and data flows in a gradient direction with the fastest decrease of the length of the data queue. The scheduling method can be well adapted to the random characteristics in the network, so that the scheduling method is widely applied to the Internet of things network.
Generally, a network architecture adopting the scheduling method comprises three layers of structures, namely a data center, edge nodes and terminals, wherein a bottom terminal receives or generates data, then the data is unloaded to the edge nodes, and the rest data and results are transmitted to the data center for centralized processing after the edge nodes perform cooperative processing. Each network node is established with a data queue, and the data queue dynamically changes based on the scheduling and processing results of data. The network nodes comprise edge nodes and terminals, and the edge nodes comprise routes, base stations, wireless access points and the like.
In the processing process, the transmission of data from the terminal to the data center requires a potential difference of a data queue between network nodes, that is, a difference value between a data accumulation amount of the terminal and a data accumulation amount of an edge node connected to the terminal needs to exceed a certain threshold value to be uploaded to the edge node connected to the terminal, and similarly, a difference value between the data accumulation amount of the edge node and other edge nodes connected to the edge node or the data accumulation amount of the data center needs to exceed a certain threshold value to be uploaded to the data center or sent to other edge nodes.
Therefore, each network node needs a large amount of data accumulation to ensure the effective operation of the network, and the terminal as the lowest layer of the network needs the most data accumulation. The storage space of the terminal is very limited, so that the problems that the network working efficiency is low and the network operation environment cannot reach the optimum due to insufficient storage space of the terminal can occur.
Disclosure of Invention
The embodiment of the invention aims to provide a data scheduling method, a data scheduling device and a data scheduling system based on the length of a data queue, so as to optimize a network operation environment and solve the problem of low network working efficiency caused by insufficient storage space of a terminal. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a data scheduling method based on a data queue length, where the method includes:
determining each destination node in a network according to a network topological structure, wherein the destination node comprises an edge node and a terminal;
determining a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0;
and setting the virtual data value of the data queue of each destination node so that the sum of the corresponding virtual data value and the actual storage data volume of each destination node is used as the total data volume of the data queue, and performing data processing based on the total data volume.
Optionally, the step of determining the virtual data value of the data queue corresponding to each destination node according to a preset scheduling algorithm includes:
and determining the virtual data values of the data processing queue and the result storage queue of the edge node according to a preset scheduling algorithm, and determining the virtual data value of the data processing queue of the terminal.
Optionally, the determining, according to a preset scheduling algorithm, virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data value of the data processing queue of the terminal includes:
carrying out repeated iterative computation on the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data value of the data processing queue of the terminal according to the following formula;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
Optionally, the method further includes:
determining a data center in a network as a source node;
and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
In a second aspect, an embodiment of the present invention provides a data scheduling system based on a data queue length, where the system includes:
the management equipment is used for determining each destination node in a network according to a network topological structure, determining a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm, and setting the virtual data value of the data queue of each destination node, wherein each destination node comprises an edge node and a terminal, and the virtual data value is greater than 0;
the terminal is used for taking the sum of the corresponding virtual data value and the actual storage data amount as the total data amount of the data queue and carrying out data processing based on the total data amount;
the edge node is used for taking the sum of the corresponding virtual data value and the actual storage data amount as the total data amount of the data queue, processing data based on the total data amount, and sending a processing result and/or data to be processed to the data center;
and the data center is used for receiving the processing result and/or the data to be processed and performing preset processing.
Optionally, the management device is specifically configured to determine, according to a preset scheduling algorithm, virtual data values of the data processing queue and the result storage queue of the edge node, and determine a virtual data value of the data processing queue of the terminal.
Optionally, the management device is specifically configured to perform multiple iterative computations according to the following formula to obtain virtual data values of the data processing queue and the result storage queue of the edge node, and virtual data values of the data processing queue of the terminal;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
Optionally, the terminal is specifically configured to use a sum of the corresponding virtual data value and an actual stored data amount as a total data amount of the data processing queue, and when a difference between the total data amount and a total data amount of the data processing queue of the edge node connected to the terminal is greater than a preset threshold, upload data in the data processing queue to the edge node connected to the terminal;
the edge node is specifically configured to acquire data uploaded by a terminal connected to the edge node, use a sum of a data amount of the data and a corresponding virtual data value as a total data amount of a data processing queue, process data in the data processing queue to obtain a processing result, use the sum of the data amount of the processing result and the corresponding virtual data value as a total data amount of a result storage queue, and upload the data in the data processing queue and/or the processing result in the result storage queue to the data center based on the total data amount of the data processing queue and the total data amount of the result storage queue.
Optionally, the management device is further configured to determine the data center as a source node; and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
In a third aspect, an embodiment of the present invention further provides a data scheduling apparatus based on a data queue length, where the apparatus includes:
the node determining module is used for determining each destination node in the network according to the network topology structure, wherein the destination node comprises an edge node and a terminal;
a virtual data value determining module, configured to determine, according to a preset scheduling algorithm, a virtual data value of a data queue corresponding to each destination node, where the virtual data value is greater than 0;
and the virtual data value setting module is used for setting the virtual data value of the data queue of each destination node so that the sum of the corresponding virtual data value and the actual storage data volume of each destination node is used as the total data volume of the data queue, and data processing is performed based on the total data volume.
In the scheme provided by the embodiment of the invention, each destination node in a network is determined according to a network topological structure, wherein the destination node comprises an edge node and a terminal; and then determining a virtual data value of the data queue corresponding to each target node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0, and then setting the virtual data value of the data queue of each target node, so that the sum of the corresponding virtual data value and the actual storage data amount is used as the total data amount of the data queue by each target node, and performing data processing based on the total data amount. Because the virtual data value of each destination node does not occupy the storage space, each destination node, particularly a terminal, can save a large amount of storage space to store actually arrived data and optimize the network operation environment, and each network node takes the sum of the virtual data value and the actually stored data amount as the total data amount of the data queue, so that data transmission to the upper layer of the network is started when the difference value between the data amount of the actually arrived data and the data accumulation amount of the destination node connected with the actually arrived data does not reach the threshold value, and the network working efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a network topology structure diagram applicable to a data scheduling method based on a data queue length according to an embodiment of the present invention;
fig. 2 is a flowchart of a data scheduling method based on a data queue length according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating a method for determining the amount of virtual data corresponding to the data center according to the embodiment shown in FIG. 2;
fig. 4 is a schematic overall structure diagram of a data scheduling method based on the length of the data queue according to the embodiment shown in fig. 2;
fig. 5 is a schematic structural diagram of a data scheduling system based on a data queue length according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a data scheduling apparatus based on a data queue length according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a management device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In order to optimize a network operation environment and solve the problem of low network working efficiency caused by insufficient terminal storage space, embodiments of the present invention provide a data scheduling method, apparatus, system, scheduling device and computer-readable storage medium based on data queue length.
First, a data scheduling method based on a data queue length provided in an embodiment of the present invention is described below.
To facilitate understanding of the data scheduling method based on the data queue length provided in the embodiment of the present invention, a network topology to which the scheme provided in the embodiment of the present invention can be applied is first described below. As shown in fig. 1, the network topology may include a terminal 101, an edge node 102, and a data center 103. The terminal 101 may be an internet of things device, a mobile phone, and the like, the edge node 102 may include a base station, a router, a wireless access point, and the like, and the data center 103 is an electronic device for data processing, and may be a processor, a server, and the like, which are not specifically limited herein.
The terminal 101 is typically communicatively connected to a single edge node, each edge node 102 may be communicatively connected to each other, and may transmit data to each other, and all edge nodes 102 are communicatively connected to the data center 103. Each terminal 101 is provided with a data processing queue 104 for storing data generated or obtained by itself, and when the total data amount of the data stored in the data processing queue 104 reaches a certain preset threshold, the terminal 101 uploads the data to the edge node 102 connected thereto.
The edge node 102 is generally provided with a data processing queue 105 and a result storage queue 106, the received data uploaded by the terminal 101 is stored in the data processing queue 105, and can process a part of the data to obtain a data result, the data result is stored in the result storage queue 106, and when the total data amount of the data stored in the data processing queue 105 or the result storage queue 106 of the edge node 102 is greater than a certain preset threshold, the data result or unprocessed data is uploaded to the data center 103 for further processing.
Based on the network topology structure, in order to implement scheduling of data processing in a network, the data scheduling method based on the data queue length provided in the embodiment of the present invention may be applied to a management device in a network, and may be a processor, a computer, a tablet computer, and the like, which is not specifically limited herein.
As shown in fig. 2, a data scheduling method based on a data queue length includes:
s201, determining each destination node in the network according to the network topology structure;
the destination node comprises an edge node and a terminal.
S202, determining virtual data values of the data queues corresponding to the target nodes according to a preset scheduling algorithm;
wherein the dummy data value is greater than 0.
And S203, setting the virtual data value of the data queue of each destination node, so that each destination node takes the sum of the corresponding virtual data value and the actual storage data amount as the total data amount of the data queue, and performing data processing based on the total data amount.
It can be seen that, in the solution provided in the embodiment of the present invention, each destination node in the network is first determined according to the network topology, where the destination node includes an edge node and a terminal; and then determining a virtual data value of the data queue corresponding to each target node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0, and then setting the virtual data value of the data queue of each target node, so that the sum of the corresponding virtual data value and the actual storage data amount is used as the total data amount of the data queue by each target node, and performing data processing based on the total data amount. Because the virtual data value of each destination node does not occupy the storage space, each destination node, particularly a terminal, can save a large amount of storage space to store actually arrived data and optimize the network operation environment, and each network node takes the sum of the virtual data value and the actually stored data amount as the total data amount of the data queue, so that data transmission to the upper layer of the network is started when the difference value between the data amount of the actually arrived data and the data accumulation amount of the destination node connected with the actually arrived data does not reach the threshold value, and the network working efficiency is improved.
In the above step S201, in order to determine each destination node in the network and its connection relationship, the management device may first determine each destination node in the network according to the network topology. The destination node may include a terminal and an edge node. For example, as shown in fig. 1, the management device may determine that each destination node in the network corresponding to the network topology is 16, including 6 terminals and 12 edge nodes.
Then, the management device may determine the virtual data value of the data queue corresponding to each of the above-mentioned destination nodes according to a preset scheduling algorithm. In order to enable the destination node to determine a total data amount greater than the data amount of the actually stored data, the virtual data value is greater than 0.
It should be noted that the virtual data value is a numerical value, is not real data, and does not occupy an actual storage space. For example, a dummy data value of 200 megabits is not data of 200 megabits in size, but rather a value of 200 megabits. In one embodiment, the virtual data values may be represented using placeholders.
Thus, after the management determines the virtual data value of each destination node, the virtual data value of the data queue of each destination node can be set. It should be noted that the virtual data values of the destination nodes may be the same or different, and the specific value management device may be determined according to factors such as an actual network topology and a data type, and is not specifically limited herein.
For example, if the management device determines that the virtual data values of the terminals 101 shown in fig. 1 are 300 megabits, 350 megabits, 380 megabits, 450 megabits, 500 megabits, and 580 megabits, respectively, in step S202, the management device may set the virtual data values of the data queues of the terminals to be 300 megabits, 350 megabits, 380 megabits, 450 megabits, 500 megabits, and 580 megabits, respectively.
Each destination node may take the sum of the virtual data value set by the management apparatus and the actual data amount of the data stored therein as a total data amount, and further perform data processing based on the total data amount. For example, assuming that the virtual data value set by the management device for the terminal a is 300 megabits and the data amount of the actually stored data is 260 megabits, the terminal a may determine that the total data amount of itself is 560 megabits, and if the total data amount of the data processing queue at the time of the edge node B connected thereto is 400 megabits, the corresponding data threshold value is 100 megabits, and since the difference between the total data amount of the terminal a itself and the total data amount of the data processing queue at the time of the edge node B exceeds 100 megabits, the terminal a may upload the data stored therein to the edge node B connected thereto. It can be seen that the amount of data actually stored by terminal a is much less than 400 megabytes, and data will not be uploaded to edge node B using the prior art scheme.
The data scheduling method based on the length of the data queue increases the potential difference between the data queues in the network, namely the difference between the total data quantity of each data queue, so that the data transmission between each data queue is faster, the efficiency is higher, and the overall working efficiency of the network is improved.
As an implementation manner of the embodiment of the present invention, the step of determining the virtual data value of the data queue corresponding to each destination node according to a preset scheduling algorithm may include:
and determining the virtual data values of the data processing queue and the result storage queue of the edge node according to a preset scheduling algorithm, and determining the virtual data value of the data processing queue of the terminal.
In order to further improve the network working efficiency, the management device of the edge node can determine the virtual data value of the result storage queue in addition to the virtual data value of the data processing queue, so that the data result in the result storage queue of the edge node can be uploaded to the data center for processing as soon as possible.
For the terminal, since the storage space of the terminal is limited and the data processing capability is also limited, the terminal generally does not process data, and a result storage queue does not exist, so that the management device only needs to determine the virtual data value of the data processing queue.
It can be seen that, in this embodiment, for the terminal, the management device only needs to determine the virtual data value of its data processing queue. For the edge node, the management device can determine the virtual data value of the result storage queue in addition to the virtual data value of the data processing queue, so that the data result in the result storage queue of the edge node can be uploaded to the data center for processing as soon as possible, and the network working efficiency is further improved.
As an implementation manner of the embodiment of the present invention, when the management device determines a virtual data value of a result storage queue in addition to a virtual data value of a data processing queue of an edge node, the determining, according to a preset scheduling algorithm, virtual data values of the data processing queue and the result storage queue of the edge node, and a virtual data value of a data processing queue of the terminal may include:
and carrying out repeated iteration calculation on the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data value of the data processing queue of the terminal according to the following formula.
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0} (1)
N_(i,j)＝V*G_(i,j)-C_(i,j) (2)
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0} (3)
H(i)＝V*p*K(i)-F(i)/p (4)
D_(m,0)＝max{Q_(i,0)+S(m),0} (5)
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T (6)
Wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j.
V in the formula (2) is a preset control parameter, and a specific value thereof may be determined according to factors such as data processing amount in an actual application scenario, and is not specifically limited herein. G (i, j) represents the minimum transmission overhead for transmitting 1-bit data from edge node i to edge node j, C (i, j) represents the maximum link capacity of the link between edge node i and edge node j, and the values of G (i, j) and C (i, j) may be determined according to actual link conditions.
It can be seen that the virtual data values of the result storage queues of each edge node in the network can be determined by the above equations (1) and (2). For convenience of calculation, default values may be set for R _ (i,0) and R _ (j,0), and may be, for example, 0, 1, 10, and the like. Thus, it can be seen from the equations (1) and (2) that R _ (i,0) and R _ (j,0) have a coupling relationship, so that after a finite number of iterations, the value of R _ (i,0) tends to converge, and the value of R _ (i,0) can be determined.
Q _ (i,0) in equation (3) represents the virtual data value of the data Processing queue of edge node i, a represents the ratio of the data amount of the result obtained by data Processing at edge node i to the data amount before Processing, Q _ (j,0) represents the virtual data value of edge node j, p in equation (4) represents the number of CPU (Central Processing Unit) cycles required to process 1-bit data, k (i) represents the computational overhead per CPU cycle of edge node i, and f (i) represents the computational power of edge node i, which may be in cycles per second of CPU operation.
A in equation (3) represents a ratio of a data amount value of a result obtained by data processing at the edge node i to a data amount before processing, that is, a percentage of a data amount of the processed data result to a data amount before processing.
It can be seen that after the above R _ (i,0) is obtained by calculation, the virtual data value of the data processing queue of each edge node can be determined according to the above formula (3) and formula (4). Similarly, for convenience of calculation, default values may be set for Q _ (i,0) and Q _ (j,0), for example, 0, 1, 10, and the like. Thus, it can be seen from equations (3) and (4) that Q _ (i,0) and Q _ (j,0) have a coupling relationship, so that after a finite number of iterations, the value of Q _ (i,0) tends to converge, and the value of Q _ (i,0) can be determined.
D _ (m,0) in formula (5) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v is the number of terminals in the network, z (m) in formula (6) represents a transmission overhead of the terminal m, Y (m, i) represents a maximum link capacity of a link between the terminal m and the edge node i, and T represents a minimum system time, that is, a time scale for each update of a system-related value.
It can be seen that after Q _ (i,0) is obtained through calculation, the virtual data value of the data processing queue of each terminal in the network can be determined through the above formula (5) and formula (6).
It can be seen that, in this embodiment, the management device may calculate the virtual data values corresponding to each terminal and each edge node in the network according to the above formulas (1) to (6), and may determine each virtual data value quickly and accurately. In addition, because the factors such as an actual network link and overhead are considered in the formulas (1) to (6), the virtual data values calculated by the formulas (1) to (6) better meet the requirements of an actual scene, the network operation environment can be optimized to the greatest extent, and the network working efficiency is improved.
As an implementation manner of the embodiment of the present invention, as shown in fig. 3, the method may further include:
s301, determining a data center in a network as a source node;
s302, determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
Since all data in the network are finally uploaded to the data center for processing, the management device may determine the data center in the network as a source node to correspond to each destination node.
Furthermore, since the data center does not have a previous network structure and does not need to upload data to other devices for processing, the data processing queue and the result storage queue of the data center do not actually need a virtual data value, and the management device can determine the virtual data values of the data processing queue and the result storage queue of the source node to be 0.
It can be seen that, in this embodiment, since the data center does not have a previous network structure, and does not need to upload data to other devices for processing, the management device may determine the virtual data values of the data processing queue and the result storage queue of the source node to be 0, so as to ensure that the data processing queue and the result storage queue of the source node can store actual data to the maximum extent, and on the other hand, ensure a network operating state, and improve network operating efficiency.
A data scheduling method based on the length of the data queue according to an embodiment of the present invention is described below with reference to fig. 4.
The network topology shown in fig. 4 includes a management device, a data center, 3 edge nodes, and a plurality of terminals, each terminal has a data processing queue, each edge node has a data processing queue and a result storage queue, and each data queue has a virtual data value set by the management device, that is, a portion indicated by a slash in the figure. The size of the area indicated by the oblique lines in the drawing is the data amount represented by the dummy data value, and is not the data amount actually occupied by the dummy data value.
The terminal scheduling, offloading policy and data processing in fig. 4 represent processing and transmission policy of data by the edge node or the data center, and the specific implementation manner is realized by the data scheduling method based on the data queue length, that is, determined according to potential differences among data queues of the terminal, the edge node and the data center.
It can be seen that when the difference between the sum of the virtual data value and the actual storage data amount in the data processing queue of each terminal and the total data amount of the data processing queue of the edge node connected to the terminal is greater than the corresponding preset threshold, the terminal uploads the data to the corresponding edge node. Similarly, when the sum of the virtual data value and the actual storage data amount in the data processing queue in the edge node and the sum of the virtual data value and the actual storage data amount in the result storage queue in the edge node satisfy the corresponding potential difference condition, the edge node uploads the unprocessed data and/or the data result to other edge nodes or the data center.
When the sum of the virtual data value in the result storage queue in the edge node and the actual storage data amount satisfies the corresponding potential difference condition, the specific manner of uploading the unprocessed data and/or the data result to other edge nodes or the data center by the edge node may be implemented by a manner in the prior art, which is not specifically limited and described herein.
It should be noted that fig. 4 only shows the data processing queue and the result storage queue in one edge node, but does not indicate that other edge nodes do not have the data processing queue and the result storage queue, each edge node in the network has the data processing queue and the result storage queue, and the management device may set a virtual data value for the data processing queue and the result storage queue of each edge node.
Corresponding to the data scheduling method based on the data queue length, the embodiment of the invention also provides a data scheduling system based on the data queue length.
The following describes a data scheduling system based on the length of a data queue according to an embodiment of the present invention.
As shown in fig. 5, a data scheduling system based on data queue length, the system includes:
the management device 510 is configured to determine each destination node in a network according to a network topology, determine a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm, and set the virtual data value of the data queue of each destination node;
the destination node comprises an edge node and a terminal, and the virtual data value is greater than 0.
A terminal 520, configured to use the sum of the corresponding virtual data value and the actual stored data amount as a total data amount of the data queue, and perform data processing based on the total data amount;
the edge node 530 is configured to use the sum of the corresponding virtual data value and the actual storage data amount as a total data amount of the data queue, perform data processing based on the total data amount, and send a processing result and/or data to be processed to the data center;
and the data center 540 is configured to receive the processing result and/or the data to be processed, and perform preset processing.
It can be seen that, in the solution provided in the embodiment of the present invention, the management device may determine each destination node in the network according to the network topology, where the destination node includes an edge node and a terminal; and then determining a virtual data value of the data queue corresponding to each destination node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0, then setting the virtual data value of the data queue of each destination node, and enabling the terminal to take the sum of the corresponding virtual data value and the actual stored data amount as the total data amount of the data queue and perform data processing based on the total data amount. The edge node may use the sum of the corresponding virtual data value and the actual storage data amount as a total data amount of the data queue, perform data processing based on the total data amount, and send a processing result and/or data to be processed to the data center. The data center can receive the processing result and/or the data to be processed and perform preset processing. Because the virtual data values of each terminal and the edge nodes do not occupy the storage space, each node, especially the terminal, can save a large amount of storage space to store the actually arrived data and optimize the network operation environment, and each network node takes the sum of the virtual data value and the actually stored data amount as the total data amount of the data queue, so that the data transmission to the upper layer of the network is started when the difference value between the data amount of the actually arrived data and the data cumulant of the destination node connected with the actually arrived data does not reach the threshold value, and the network work efficiency is improved.
As an implementation manner of the embodiment of the present invention, the management device 510 may be specifically configured to determine, according to a preset scheduling algorithm, virtual data values of a data processing queue and a result storage queue of the edge node, and determine a virtual data value of a data processing queue of the terminal.
As an implementation manner of the embodiment of the present invention, the management device 510 may be specifically configured to perform multiple iterations according to the following formula to calculate virtual data values of the data processing queue and the result storage queue of the edge node and virtual data values of the data processing queue of the terminal;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
As an implementation manner of the embodiment of the present invention, the terminal 520 may be specifically configured to use a sum of the corresponding virtual data value and the actual stored data amount as a total data amount of the data processing queue, and when a difference between the total data amount and the total data amount of the data processing queue of the edge node connected to the terminal is greater than a preset threshold, upload data in the data processing queue to the edge node 530 connected to the terminal;
the edge node 530 may be specifically configured to acquire data uploaded by the terminal 520 connected to the edge node, use a sum of a data amount of the data and a corresponding virtual data value as a total data amount of a data processing queue, process data in the data processing queue to obtain a processing result, use a sum of a data amount of the processing result and a corresponding virtual data value as a total data amount of a result storage queue, and upload the data in the data processing queue and/or the processing result in the result storage queue to the data center 540 based on the total data amount of the data processing queue and the total data amount of the result storage queue.
The preset threshold may be determined according to actual factors such as an actual terminal and a storage space of an edge node, and is not specifically limited herein.
As an implementation manner of the embodiment of the present invention, the management device 510 may be further configured to determine the data center 540 as a source node; and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
Corresponding to the data scheduling method based on the data queue length, the embodiment of the invention also provides a data scheduling device based on the data queue length.
The following describes a data scheduling apparatus based on data queue length according to an embodiment of the present invention.
As shown in fig. 6, an apparatus for scheduling data based on a data queue length, the apparatus comprising:
a node determining module 610, configured to determine each destination node in the network according to the network topology;
the destination node comprises an edge node and a terminal.
A virtual data value determining module 620, configured to determine, according to a preset scheduling algorithm, a virtual data value of a data queue corresponding to each destination node;
wherein the dummy data value is greater than 0.
A virtual data value setting module 630, configured to set a virtual data value of the data queue of each destination node, so that the destination node uses a sum of the corresponding virtual data value and an actual storage data amount as a total data amount of the data queue, and performs data processing based on the total data amount.
It can be seen that, in the solution provided in the embodiment of the present invention, each destination node in the network is first determined according to the network topology, where the destination node includes an edge node and a terminal; and then determining a virtual data value of the data queue corresponding to each target node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0, and then setting the virtual data value of the data queue of each target node, so that the sum of the corresponding virtual data value and the actual storage data amount is used as the total data amount of the data queue by each target node, and performing data processing based on the total data amount. Because the virtual data value of each destination node does not occupy the storage space, each destination node, particularly a terminal, can save a large amount of storage space to store actually arrived data and optimize the network operation environment, and each network node takes the sum of the virtual data value and the actually stored data amount as the total data amount of the data queue, so that data transmission to the upper layer of the network is started when the difference value between the data amount of the actually arrived data and the data accumulation amount of the destination node connected with the actually arrived data does not reach the threshold value, and the network working efficiency is improved.
As an implementation manner of the embodiment of the present invention, the virtual data value determining module 620 may include:
a virtual data value determining unit (not shown in fig. 6) configured to determine, according to a preset scheduling algorithm, virtual data values of the data processing queue and the result storage queue of the edge node, and determine a virtual data value of the data processing queue of the terminal.
As an implementation manner of the embodiment of the present invention, the virtual data value determining unit may include:
a virtual data value determining subunit (not shown in fig. 6) configured to perform multiple iterations to calculate virtual data values of the data processing queue and the result storage queue of the edge node and virtual data values of the data processing queue of the terminal according to the following formulas;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
As an implementation manner of the embodiment of the present invention, the apparatus may further include:
a source node determining module (not shown in fig. 6) for determining a data center in the network as a source node;
and a source node virtual data value determining module (not shown in fig. 6) configured to determine that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
As shown in fig. 7, the controller may include a processor 701, a communication interface 702, a memory 703 and a communication bus 704, where the processor 701, the communication interface 702 and the memory 703 complete communication with each other through the communication bus 704,
a memory 703 for storing a computer program;
the processor 701 is configured to implement the following steps when executing the program stored in the memory 703:
determining each destination node in the network according to the network topology structure;
the destination node comprises an edge node and a terminal.
Determining a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm;
wherein the dummy data value is greater than 0.
And setting the virtual data value of the data queue of each destination node so that the sum of the corresponding virtual data value and the actual storage data volume of each destination node is used as the total data volume of the data queue, and performing data processing based on the total data volume.
It can be seen that, in the solution provided in the embodiment of the present invention, the management device first determines each destination node in the network according to the network topology, where the destination node includes an edge node and a terminal; and then determining a virtual data value of the data queue corresponding to each target node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0, and then setting the virtual data value of the data queue of each target node, so that the sum of the corresponding virtual data value and the actual storage data amount is used as the total data amount of the data queue by each target node, and performing data processing based on the total data amount. Because the virtual data value of each destination node does not occupy the storage space, each destination node, particularly a terminal, can save a large amount of storage space to store actually arrived data and optimize the network operation environment, and each network node takes the sum of the virtual data value and the actually stored data amount as the total data amount of the data queue, so that data transmission to the upper layer of the network is started when the difference value between the data amount of the actually arrived data and the data accumulation amount of the destination node connected with the actually arrived data does not reach the threshold value, and the network working efficiency is improved.
The communication bus mentioned in the management apparatus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the management device and other devices.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The step of determining the virtual data value of the data queue corresponding to each destination node according to a preset scheduling algorithm may include:
and determining the virtual data values of the data processing queue and the result storage queue of the edge node according to a preset scheduling algorithm, and determining the virtual data value of the data processing queue of the terminal.
The determining the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data values of the data processing queue of the terminal according to the preset scheduling algorithm may include:
carrying out repeated iterative computation on the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data value of the data processing queue of the terminal according to the following formula;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
Wherein, the method can also comprise:
determining a data center in a network as a source node;
and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when executed by a processor, the computer program implements the following steps:
determining each destination node in the network according to the network topology structure;
the destination node comprises an edge node and a terminal.
Determining a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm;
wherein the dummy data value is greater than 0.
And setting the virtual data value of the data queue of each destination node so that the sum of the corresponding virtual data value and the actual storage data volume of each destination node is used as the total data volume of the data queue, and performing data processing based on the total data volume.
It can be seen that, in the solution provided in the embodiment of the present invention, when the computer program is executed by the processor, each destination node in the network is first determined according to the network topology, where the destination node includes an edge node and a terminal; and then determining a virtual data value of the data queue corresponding to each target node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0, and then setting the virtual data value of the data queue of each target node, so that the sum of the corresponding virtual data value and the actual storage data amount is used as the total data amount of the data queue by each target node, and performing data processing based on the total data amount. Because the virtual data value of each destination node does not occupy the storage space, each destination node, particularly a terminal, can save a large amount of storage space to store actually arrived data and optimize the network operation environment, and each network node takes the sum of the virtual data value and the actually stored data amount as the total data amount of the data queue, so that data transmission to the upper layer of the network is started when the difference value between the data amount of the actually arrived data and the data accumulation amount of the destination node connected with the actually arrived data does not reach the threshold value, and the network working efficiency is improved.
The step of determining the virtual data value of the data queue corresponding to each destination node according to a preset scheduling algorithm may include:
and determining the virtual data values of the data processing queue and the result storage queue of the edge node according to a preset scheduling algorithm, and determining the virtual data value of the data processing queue of the terminal.
The determining the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data values of the data processing queue of the terminal according to the preset scheduling algorithm may include:
carrying out repeated iterative computation on the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data value of the data processing queue of the terminal according to the following formula;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
Wherein, the method can also comprise:
determining a data center in a network as a source node;
and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
It should be noted that, for the above system, apparatus, management device and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is simple, and the relevant points can be referred to the partial description of the method embodiments.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A data scheduling method based on data queue length is characterized in that the method comprises the following steps:
determining each destination node in a network according to a network topological structure, wherein the destination node comprises an edge node and a terminal;
determining a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm, wherein the virtual data value is greater than 0;
and setting the virtual data value of the data queue of each destination node so that the sum of the corresponding virtual data value and the actual storage data volume of each destination node is used as the total data volume of the data queue, and performing data processing based on the total data volume.
2. The method of claim 1, wherein the step of determining the virtual data value of the data queue corresponding to each destination node according to a preset scheduling algorithm comprises:
and determining the virtual data values of the data processing queue and the result storage queue of the edge node according to a preset scheduling algorithm, and determining the virtual data value of the data processing queue of the terminal.
3. The method of claim 2, wherein the steps of determining the virtual data values of the data processing queue and the result storage queue of the edge node and determining the virtual data values of the data processing queue of the terminal according to a predetermined scheduling algorithm comprise:
carrying out repeated iterative computation on the virtual data values of the data processing queue and the result storage queue of the edge node and the virtual data value of the data processing queue of the terminal according to the following formula;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
4. The method of any one of claims 1-3, further comprising:
determining a data center in a network as a source node;
and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
5. A data scheduling system based on data queue length, the system comprising:
the management equipment is used for determining each destination node in a network according to a network topological structure, determining a virtual data value of a data queue corresponding to each destination node according to a preset scheduling algorithm, and setting the virtual data value of the data queue of each destination node, wherein each destination node comprises an edge node and a terminal, and the virtual data value is greater than 0;
the terminal is used for taking the sum of the corresponding virtual data value and the actual storage data amount as the total data amount of the data queue and carrying out data processing based on the total data amount;
the edge node is used for taking the sum of the corresponding virtual data value and the actual storage data amount as the total data amount of the data queue, processing data based on the total data amount, and sending a processing result and/or data to be processed to the data center;
and the data center is used for receiving the processing result and/or the data to be processed and performing preset processing.
6. The system of claim 5,
the management device is specifically configured to determine, according to a preset scheduling algorithm, virtual data values of the data processing queue and the result storage queue of the edge node, and determine a virtual data value of the data processing queue of the terminal.
7. The system of claim 5,
the management device is specifically configured to perform multiple iterative computations according to the following formulas to obtain virtual data values of the data processing queue and the result storage queue of the edge node and virtual data values of the data processing queue of the terminal;
R_(i,0)＝max{min_j{R_(j,0)+N_(i,j)},0}；
N_(i,j)＝V*G_(i,j)-C_(i,j)；
Q_(i,0)＝max{min_j{a*R_(i,0)+H(i)，Q_(j,0)+N(i,j)}，0}；
H(i)＝V*p*K(i)-F(i)/p；
D_(m,0)＝max{Q_(i,0)+S(m),0}；
S(m)＝V*Z(m)/Y(m,i)-Y(m,i)*T；
wherein, R _ (i,0) represents the virtual data value of the result storage queue of the edge node i belonging to {1,2.. u }, u is the number of edge nodes in the network, R _ (j,0) represents the virtual data value of the result storage queue of the edge node j belonging to {1,2.. u } connected with the edge node i, V is a preset control parameter, G _ (i, j) represents the minimum transmission overhead of transmitting 1-bit data from the edge node i to the edge node j, and C _ (i, j) represents the maximum link capacity of the link between the edge node i and the edge node j; q _ (i,0) represents a virtual data value of the data processing queue of the edge node i, a represents a ratio of a data quantity value of a result obtained through data processing at the edge node i to a data quantity before processing, Q _ (j,0) represents a virtual data value of the edge node j, p represents a number of CPU cycles required for processing 1-bit data, k (i) represents a calculation overhead per CPU cycle of the edge node i, and f (i) represents a calculation capability of the edge node i; d _ (m,0) represents a virtual data value of a data processing queue of a terminal m e {1,2.. v } connected to the edge node i, v being the number of terminals in the network, z (m) represents the transmission overhead of the terminal m, Y (m, i) represents the maximum link capacity of the link between the terminal m and the edge node i, and T represents the minimum system time.
8. The system of claim 7,
the terminal is specifically configured to use the sum of the corresponding virtual data value and the actual stored data amount as a total data amount of the data processing queue, and when a difference between the total data amount and a total data amount of the data processing queue of the edge node connected to the terminal is greater than a preset threshold, upload data in the data processing queue to the edge node connected to the terminal;
the edge node is specifically configured to acquire data uploaded by a terminal connected to the edge node, use a sum of a data amount of the data and a corresponding virtual data value as a total data amount of a data processing queue, process data in the data processing queue to obtain a processing result, use the sum of the data amount of the processing result and the corresponding virtual data value as a total data amount of a result storage queue, and upload the data in the data processing queue and/or the processing result in the result storage queue to the data center based on the total data amount of the data processing queue and the total data amount of the result storage queue.
9. The system of any one of claims 5-8,
the management device is further configured to determine the data center as a source node; and determining that the virtual data values of the data processing queue and the result storage queue of the source node are both 0.
10. An apparatus for scheduling data based on a length of a data queue, the apparatus comprising:
the node determining module is used for determining each destination node in the network according to the network topology structure, wherein the destination node comprises an edge node and a terminal;
a virtual data value determining module, configured to determine, according to a preset scheduling algorithm, a virtual data value of a data queue corresponding to each destination node, where the virtual data value is greater than 0;
and the virtual data value setting module is used for setting the virtual data value of the data queue of each destination node so that the sum of the corresponding virtual data value and the actual storage data volume of each destination node is used as the total data volume of the data queue, and data processing is performed based on the total data volume.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811054783.6A CN109347763B (en) | 2018-09-11 | 2018-09-11 | Data scheduling method, device and system based on data queue length |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811054783.6A CN109347763B (en) | 2018-09-11 | 2018-09-11 | Data scheduling method, device and system based on data queue length |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109347763A CN109347763A (en) | 2019-02-15 |
| CN109347763B true CN109347763B (en) | 2021-11-05 |
Family
ID=65304860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811054783.6A Active CN109347763B (en) | 2018-09-11 | 2018-09-11 | Data scheduling method, device and system based on data queue length |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109347763B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109981487B (en) * | 2019-03-27 | 2020-10-27 | 北京邮电大学 | Data scheduling method and device, electronic equipment and readable storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1805308A (en) * | 2005-01-12 | 2006-07-19 | 日本电气株式会社 | Radio packet signal transmission system, radio packet signal transmission terminal and radio packet signal transmission method used in these |
| EP1696339A2 (en) * | 2005-02-25 | 2006-08-30 | Microsoft Corporation | System and method for testing data format using targeted variant input |
| CN102630378A (en) * | 2009-11-25 | 2012-08-08 | 瑞典爱立信有限公司 | Individualized data communication on the basis of a descriptive file |
| CN102740467A (en) * | 2011-04-02 | 2012-10-17 | 华为技术有限公司 | Uplink feedback method of high speed dedicated physical control channel (HS-DPCCH), terminal and base station thereof |
| CN103188801A (en) * | 2011-12-29 | 2013-07-03 | 中兴通讯股份有限公司 | Control method and control device of station (STA) |
| CN104066122A (en) * | 2014-07-09 | 2014-09-24 | 东南大学 | Congestion control and transmission scheduling method for cellular and D2D hybrid network |
| WO2016020726A1 (en) * | 2014-08-07 | 2016-02-11 | Telefonaktiebolaget L M Ericsson (Publ) | Data transfer in a system of connected things |
| CN105573662A (en) * | 2014-10-31 | 2016-05-11 | 群联电子股份有限公司 | Data writing method, storage control circuit unit and storage storing device |
| CN105872003A (en) * | 2015-12-21 | 2016-08-17 | 乐视云计算有限公司 | Data processing method, device and system based on P2P (Peer-to-Peer) network |
| CN106612335A (en) * | 2017-02-07 | 2017-05-03 | 济南浪潮高新科技投资发展有限公司 | Method of adopting Docker container to realize IoT (Internet of things) information exchange and communication |
| CN108959313A (en) * | 2017-05-23 | 2018-12-07 | 中国移动通信集团重庆有限公司 | Concurrent processing method, device and storage medium towards mass small documents |
-
2018
- 2018-09-11 CN CN201811054783.6A patent/CN109347763B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1805308A (en) * | 2005-01-12 | 2006-07-19 | 日本电气株式会社 | Radio packet signal transmission system, radio packet signal transmission terminal and radio packet signal transmission method used in these |
| EP1696339A2 (en) * | 2005-02-25 | 2006-08-30 | Microsoft Corporation | System and method for testing data format using targeted variant input |
| CN102630378A (en) * | 2009-11-25 | 2012-08-08 | 瑞典爱立信有限公司 | Individualized data communication on the basis of a descriptive file |
| CN102740467A (en) * | 2011-04-02 | 2012-10-17 | 华为技术有限公司 | Uplink feedback method of high speed dedicated physical control channel (HS-DPCCH), terminal and base station thereof |
| CN103188801A (en) * | 2011-12-29 | 2013-07-03 | 中兴通讯股份有限公司 | Control method and control device of station (STA) |
| CN104066122A (en) * | 2014-07-09 | 2014-09-24 | 东南大学 | Congestion control and transmission scheduling method for cellular and D2D hybrid network |
| WO2016020726A1 (en) * | 2014-08-07 | 2016-02-11 | Telefonaktiebolaget L M Ericsson (Publ) | Data transfer in a system of connected things |
| CN105573662A (en) * | 2014-10-31 | 2016-05-11 | 群联电子股份有限公司 | Data writing method, storage control circuit unit and storage storing device |
| CN105872003A (en) * | 2015-12-21 | 2016-08-17 | 乐视云计算有限公司 | Data processing method, device and system based on P2P (Peer-to-Peer) network |
| CN106612335A (en) * | 2017-02-07 | 2017-05-03 | 济南浪潮高新科技投资发展有限公司 | Method of adopting Docker container to realize IoT (Internet of things) information exchange and communication |
| CN108959313A (en) * | 2017-05-23 | 2018-12-07 | 中国移动通信集团重庆有限公司 | Concurrent processing method, device and storage medium towards mass small documents |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109347763A (en) | 2019-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104901989B (en) | A kind of Site Service offer system and method | |
| CN107454019B (en) | Dynamic bandwidth allocation method, device, equipment and storage medium for software defined network | |
| CN110365748A (en) | Treating method and apparatus, storage medium and the electronic device of business datum | |
| CN104734964A (en) | Message processing method, node and system | |
| CN111083050B (en) | Data stream transmission method and device based on software defined network | |
| CN110311863B (en) | Routing path determination method and device | |
| CN114286413A (en) | TSN network combined routing and stream distribution method and related equipment | |
| WO2022068192A1 (en) | Relay transmission scheduling method and apparatus, storage medium, and electronic device | |
| CN112383485B (en) | Network congestion control method and device | |
| CN105099908A (en) | Path calculating method, message responding method and related equipment | |
| CN109347763B (en) | Data scheduling method, device and system based on data queue length | |
| CN112256413A (en) | Scheduling method and device for edge computing task based on Internet of things | |
| CN109922161B (en) | Content distribution method, system, device and medium for dynamic cloud content distribution network | |
| JP6131203B2 (en) | Device selection network system, device selection server and program | |
| CN108347377B (en) | Data forwarding method and device | |
| CN113746763B (en) | Data processing method, device and equipment | |
| CN109981487B (en) | Data scheduling method and device, electronic equipment and readable storage medium | |
| CN110113269B (en) | Flow control method based on middleware and related device | |
| CN115809148B (en) | Load balancing task scheduling method and device for edge computing | |
| CN114422438B (en) | Link adjustment method and device for power communication network | |
| TWI488473B (en) | Auto configuration server and management method for customer premises equipments thereof | |
| CN110636471A (en) | Monitoring system based on WSN technology | |
| CN113472591A (en) | Method and device for determining service performance | |
| CN111542052B (en) | Group trust degree-based data forwarding method and device in Internet of things | |
| CN112217740B (en) | Request processing method and device based on edge computing network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |