CN114050961B - Large-scale network simulation system and resource dynamic scheduling and distributing method - Google Patents
Large-scale network simulation system and resource dynamic scheduling and distributing method Download PDFInfo
- Publication number
- CN114050961B CN114050961B CN202111314346.5A CN202111314346A CN114050961B CN 114050961 B CN114050961 B CN 114050961B CN 202111314346 A CN202111314346 A CN 202111314346A CN 114050961 B CN114050961 B CN 114050961B
- Authority
- CN
- China
- Prior art keywords
- network
- simulation
- node
- virtual
- plane
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 208
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000013507 mapping Methods 0.000 claims abstract description 76
- 239000011159 matrix material Substances 0.000 claims abstract description 29
- 238000012360 testing method Methods 0.000 claims abstract description 12
- 230000006870 function Effects 0.000 claims description 39
- 238000012549 training Methods 0.000 claims description 32
- 239000003795 chemical substances by application Substances 0.000 claims description 20
- 230000002787 reinforcement Effects 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 17
- 238000004891 communication Methods 0.000 claims description 16
- 230000007613 environmental effect Effects 0.000 claims description 8
- 239000002904 solvent Substances 0.000 claims description 8
- 238000011478 gradient descent method Methods 0.000 claims description 6
- 230000007774 longterm Effects 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- 230000008901 benefit Effects 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012800 visualization Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000005094 computer simulation Methods 0.000 description 2
- 230000008846 dynamic interplay Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
- H04L41/046—Network management architectures or arrangements comprising network management agents or mobile agents therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
- H04L41/082—Configuration setting characterised by the conditions triggering a change of settings the condition being updates or upgrades of network functionality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
- H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a large-scale network simulation system and a resource dynamic scheduling allocation method, wherein the simulation system comprises a simulation scene logic plane, a simulation control plane, a simulation data plane, a simulation storage plane and a simulation decision plane; the simulation scene logic plane generates an underlying network model according to an actual service scene, abstracts a service request of a user into a virtual network model and sends the virtual network model to a simulation decision plane; the simulation decision plane combines the virtual network request test set and the underlying network model to train a virtual node mapping model and train a virtual link mapping model; then, generating a decision for the virtual network request according to the latest underlying network resource state matrix of the simulation storage plane, and sending a decision scheme to the simulation control plane; the simulation control plane updates the resource state matrix of the underlying network according to the mapping scheme, and stores the latest resource state to the simulation storage plane, so that the problems of dynamic scheduling and management of large-scale network resources can be effectively solved.
Description
Technical Field
The invention relates to the technical field of communication network simulation, in particular to a large-scale network simulation system and a resource dynamic scheduling and distributing method.
Background
The existing network simulation means mainly comprise three types of simulation software based on discrete events, such as NS-3, OPNET and the like, a physical simulation experiment bed and a network simulation platform based on virtualization. Network simulation software such as NS3 is more vivid, but the real-time performance of simulation cannot be guaranteed by an event-driven mode, and the provided model only supports the research on the mainstream protocol and cannot test and evaluate new technologies. The test experiment bed can realize vivid network simulation experiment, but the simulation experiment bed is not only high in cost and difficult to construct, but also poor in flexibility, and can not meet the reproducibility of the experiment. The network simulation platform based on virtualization can guarantee the authenticity of simulation, generate real and interactive flow, simultaneously can meet experimental scenes of different scales, and has the characteristics of flexibility and expandability.
On the other hand, the simulation platform based on the network virtualization technology realizes the sharing of bottom resources, and the simulation platform based on the virtualization is easy to improve the simulation scale of the network due to the distributed architecture and the expandable characteristic. However, with the expansion of network scale, the service requirements in the network become more complex and changeable, and in order to support heterogeneous virtual network requests as much as possible by the underlying network, the underlying network resources need to be reasonably allocated by means of a virtual network mapping algorithm, so as to realize efficient scheduling and management of the resources.
Although the existing large-scale network simulation platform can simulate a network scene more vividly, most simulation platforms do not dynamically feed back simulation results to a simulation network, and the dynamic adaptability of the simulation network is difficult to embody. The invention introduces the reinforcement learning technology into the virtualization simulation platform, dynamically updates the decision strategy of the reinforcement learning agent network and the resource state of the simulation network through the dynamic interaction of the reinforcement learning model and the simulation network, realizes the dynamic simulation of the large-scale network based on resource scheduling, and solves the problems of the dynamic scheduling and management of the large-scale network resources.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems in the background technology, the invention provides a large-scale network simulation system and a resource dynamic scheduling allocation method, which are used for solving the problems that the simulation process of the existing simulation platform cannot form a closed loop, the historical simulation result cannot be dynamically acted into the simulation network by the simulation platform, the dynamic adaptability of the simulation network is difficult to embody, most of the simulation platforms lack a decision part, the decision process of the real network to the network change is not considered, and more complicated and changeable network scenes are difficult to simulate.
The technical scheme is as follows: in order to realize the purpose, the invention adopts the technical scheme that:
a large-scale network simulation system comprises a simulation scene logic plane, a simulation control plane, a simulation data plane, a simulation storage plane and a simulation decision plane;
the simulation scene logic plane generates a corresponding underlying network model according to an actual scene, and the underlying network model is used for simulating a real network in the actual scene; the underlying network model comprises network nodes, links associated with the network nodes, network topology relations, software systems used by the network nodes and protocol stacks; the logic plane of the simulation scene counts the service requests of the users at a certain moment, and all the service requests at a certain moment can be abstracted into a virtual network because the corresponding calculation resources and communication resources of the underlying network nodes are needed for responding to a certain service request. The node values and link values in the virtual network topology represent underlying network computing resources and communication resources required to respond to the corresponding service request, respectively. The process of mapping the virtual network to the underlying network is the process of scheduling and distributing underlying network resources; specifically, the following are shown:
G V →G S
the virtual network is denoted G V ＝(N V ,L V ),N V Representing virtual network nodes, L V Representing a virtual network link; the underlying network is denoted G S ＝(N S ,L S )，N S Indicating underlying network nodes, L S Representing an underlying network path;
the simulation control plane comprises an SDN controller and a node controller; the SDN controller is used for maintaining a topological structure of a simulation network in a simulation data plane and controlling topological linkage of the simulation network by dynamically updating flow table contents of an SDN switch; the node controller is used for issuing a service simulation instruction and monitoring the resource attribute of each node; the node controller dynamically switches the simulation service types of the simulation network nodes according to the decision result of the simulation decision plane;
the simulation data plane constructs the same number of simulation nodes according to the data of the underlying network model, and the resource attributes of the simulation nodes, the software system and the protocol stack used by the simulation nodes are consistent with the underlying network model and are used for instantiating the underlying network model generated in the simulation scene logic plane and simulating real network nodes; the simulation data plane simulates wireless links among the nodes according to the model data of the underlying network, and the uplink and downlink bandwidths, the error rate and the packet loss rate of the links among the nodes are consistent with those of the underlying network model; the simulation data plane supports communication between the virtual nodes and the physical nodes;
the simulation storage plane captures and stores node flow information, network situation information and resource occupation information generated by the simulation data plane in the simulation process, and transmits the node flow information, the network situation information and the resource occupation information to the simulation decision plane for decision making and integration links; the simulation storage plane changes an information collection task in real time through monitoring an information collection item, and meanwhile, the simulation storage plane comprises a visualization module used for inquiring and retrieving historical node information stored in the simulation storage plane;
and the simulation decision plane receives the virtual network request of the simulation scene logic plane, generates a decision scheme, sends a decision result to the simulation control plane, and updates the resource state of the simulation data plane.
A dynamic scheduling and distributing method for the underlying network resources by adopting the large-scale network simulation system comprises the following steps:
s1, a logic plane of a simulation scene counts user service requests at a certain moment, and all service requests at a certain moment can be abstracted into a virtual network because corresponding underlying network node computing resources and communication resources are needed for responding to a certain service request. CPU resources required to be consumed by each node in the virtual network and bandwidth values of adjacent links represent underlying network computing resources and communication resources required by a certain service; sending the virtual network request to a simulation decision plane;
s2, the simulation control plane issues a control command to the simulation data plane according to the underlying network topology data provided by the simulation scene logic plane, instantiates the underlying network, and simultaneously the simulation storage plane updates the underlying network node state matrix M at the current moment n And underlying network link state matrix M p ；
S3, a simulation decision plane needs to combine a virtual network request test set and an underlying network model to train a virtual node mapping model; sequentially defining environment state parameters of the underlying network nodes, an underlying network resource matrix and a reward function by adopting a reinforcement learning method, training a reinforcement learning agent network by using a gradient descent method, then sending a virtual network request test set and underlying network topology into a virtual node mapping model for training, and training the reinforcement learning agent network according to the reward function value of each iteration;
s4, training the virtual link mapping model again on the basis of finishing the training of the virtual node mapping model; sending the virtual network request test set, the underlying network and the trained virtual node mapping model into a virtual link mapping model, and training a virtual link agent decision network by using a gradient descent method;
s5, extracting an environment state matrix of the simulation storage plane by the simulation decision plane, and sequentially performing node mapping and link mapping on the virtual network request by the reinforcement learning agent network of the simulation decision plane according to the latest environment state matrix; when the virtual network request cannot complete mapping, the simulation decision plane rejects the request and informs the user;
s6, when the simulation decision plane successfully answers a virtual network request, the decision result is sent to the simulation control plane, the simulation control plane issues a control command accordingly, the simulation data plane updates the resource state of the underlying network, and finally the updated underlying network node state matrix M is used for updating the resource state of the underlying network n And underlying network link state matrix M p And storing to the simulation storage plane.
Further, the training of the virtual node mapping model in step S3 specifically includes:
s3.1, training an agent network by using a strategy gradient method, and adjusting a mapping strategy according to feedback of an environment, namely an incentive function, wherein the higher the environment incentive value is, the better the mapping scheme is; setting the current overall profit-to-cost ratio as a training reward function, and obtaining a group of complete virtual node mapping schemes in each iteration, wherein the aim is to maximize the long-term accumulated profit; the yield is the sum of the CPU values successfully mapped to the underlying network by the virtual nodes, and the cost is the underlying network resources consumed by mapping the virtual network request, namely:
wherein R is 1 Representing the above gain, the function CpuSum (n) v ) CPU value n for calculating success of a virtual node on a bottom node v Representing a respective virtual node; c 1 Representing the above cost, the function CpuCost (n) v ) Underlying network resources, n, for calculating consumption of a virtual node in a mapping process v Representing the corresponding virtual node.
The reward function is expressed as follows:
s3.2, selecting four parameters of computing resources, node link resources, node degree centrality and approximate centrality of each node to carry out node mapping strategy optimization of the proxy network; in particular, the amount of the solvent to be used,
(1) And computing resources: with C i To indicate the currently available CPU resources of the ith node on the underlying network:
wherein the function CPU () is used to calculate the remaining available CPU resources of the current node,
representing the corresponding underlying node.
(2) Node link resource: BW (l) is the bandwidth of link l of the ith node, and the bandwidths of all the adjacent links of node i are added together to use BW i Node link resources representing the ith node on the underlying network:
(3) Node degree centrality: by D i Reflecting the number of adjacent links owned by a node:
wherein
The degree of the ith node is, and m is the number of the nodes of the underlying network;
(4) And the centering property: this attribute reflects the proximity of one node to other nodes, denoted J i Represents:
wherein d is ij Representing the number of edges on the shortest path between the node i and the node j;
after extracting the environmental state parameters of the nodes, representing the characteristic vector as V for the ith node of the underlying network i ＝(C i ,BW i ,D i ,J i ) T The node state matrix of the underlying network is denoted as M n ＝(V 1 ,V 2 ,…,V N ) T 。
Further, the training of the virtual link mapping model in step S4 specifically includes:
s4.1, training the agent network by using a strategy gradient method, and adjusting a mapping strategy according to feedback of an environment, namely an incentive function, wherein the higher the environment incentive value is, the better the mapping scheme is; setting the current overall profit-to-cost ratio as a training reward function, and obtaining a group of complete virtual node mapping schemes in each iteration, wherein the aim is to maximize the long-term accumulated profit; the profit is the sum of the link bandwidth values successfully mapped to the underlying network by the virtual network; the cost is the underlying network resources consumed by mapping the virtual network request, i.e.:
wherein R is 2 Representing the above benefit, function BWSum (l) v ) For calculating the bandwidth value, l, of a successful link to the underlying network v Representing the corresponding virtual link; c 2 Representing the above cost, function BWCost (l) v ) Underlying network resources, hop (l), used to compute the consumption of a virtual link in the mapping process v ) Indicates the number of the bottom layer links occupied by a virtual link, l v Representing the corresponding virtual link.
The reward function is expressed as follows:
s4.2, when the virtual link is mapped to the bottom link, the link segmentation condition exists; the reachable path characteristics between node pairs in the underlying network are used as environmental state parameters, and two parameters of bandwidth and medium centrality are selected for each underlying path to be optimized; in particular, the amount of the solvent to be used,
(1) And bandwidth: for all links between certain underlying paths, selecting the smallest link bandwidth value as the bandwidth of the physical path, that is:
wherein,
indicating the kth underlay path of the underlay network.
(2) The medium centrality is as follows: for all links in the k-th underlying path, use d l Representing the number of shortest paths through link l, by D k Representing the number of all shortest paths passing through the k-th underlying path, the intermediary centrality of the k-th underlying path is represented as:
wherein count represents the number of shortest path edges on path k;
after extracting the environmental state parameters of the underlying network path, expressing the characteristic vector of the k-th underlying network path as
The link state matrix of the underlying network is denoted as M p ＝(P 1 ,P 2 ,…,P N ) T 。
Has the beneficial effects that:
the large-scale network simulation system and the resource dynamic scheduling and distributing method provided by the invention introduce the reinforcement learning technology into the virtualization simulation platform, dynamically update the decision strategy of the reinforcement learning agent network and the resource state of the simulation network through the dynamic interaction of the reinforcement learning model and the simulation network, and make an active decision on a service request. On one hand, the dynamic simulation of the large-scale network based on resource scheduling is realized, and the problems of dynamic scheduling and management of large-scale network resources are solved; on the other hand, the simulation system supports simulation of more complex and real network scenes, can actively make decisions on network changes, and overcomes the defect that the conventional simulation platform lacks active decisions.
Drawings
FIG. 1 is a schematic diagram of a large-scale network simulation platform system architecture provided by the present invention;
FIG. 2 is a functional diagram of a large-scale network simulation platform system provided by the present invention;
FIG. 3 is a schematic diagram of virtual network mapping to an underlying network in an embodiment of the invention;
FIG. 4 is a diagram of a virtual node mapping model of a simulation decision plane in an embodiment of the present invention;
FIG. 5 is a diagram of a virtual link mapping model of an emulation decision plane in an embodiment of the present invention;
Detailed Description
The invention will be further described with reference to the following drawings and specific embodiments.
The invention firstly provides a large-scale network simulation system based on resource scheduling, which has a specific architecture as shown in figure 1 and a specific functional schematic diagram as shown in figure 2 and comprises a simulation scene logic plane, a simulation control plane, a simulation data plane, a simulation storage plane and a simulation decision plane.
The simulation scene logic plane generates a corresponding underlying network model according to an actual scene, and the underlying network model is used for simulating a real network in the actual scene; the underlying network model comprises network nodes, links associated with the network nodes, a network topology relation, a software system used by the network nodes and a protocol stack; the simulation scene logic plane counts the service requests of users at a certain time, and all the service requests at a certain time can be abstracted into a virtual network because corresponding underlying network nodes are required to calculate resources and communication resources when responding to a certain service request. The node values and link values in the virtual network topology represent underlying network computing resources and communication resources required to respond to the corresponding service request, respectively. The process of mapping the virtual network to the underlying network, that is, the process of scheduling and allocating the underlying network resources, is illustrated in fig. 3; specifically, the following are shown:
G V →G S
the virtual network is denoted G V ＝(N V ,L V ),N V Representing virtual network nodes, L V Representing a virtual network link; the underlying network is denoted G S ＝(N S ,L S )，N S Indicating underlying network nodes, L S Representing an underlying network path;
the simulation control plane comprises an SDN controller and a node controller; the SDN controller is used for maintaining a topological structure of a simulation network in a simulation data plane and controlling topological linkage of the simulation network by dynamically updating flow table contents of an SDN switch; the node controller is used for issuing a service simulation instruction and monitoring the resource attribute of each node; the node controller dynamically switches the simulation service types of the simulation network nodes according to the decision result of the simulation decision plane;
the simulation data plane constructs the same number of simulation nodes according to the data of the underlying network model, and the resource attributes of the simulation nodes, the software system and the protocol stack used by the simulation nodes are consistent with the underlying network model and are used for instantiating the underlying network model generated in the simulation scene logic plane and simulating real network nodes; the simulation data plane simulates wireless links among the nodes according to the model data of the underlying network, and the uplink and downlink bandwidths, the error rate and the packet loss rate of the links among the nodes are consistent with those of the underlying network model; the simulation data plane supports communication between the virtual nodes and the physical nodes;
the simulation storage plane captures and stores node flow information, network situation information and resource occupation information generated by the simulation data plane in the simulation process, and transmits the node flow information, the network situation information and the resource occupation information to the simulation decision plane for decision making and integration links; the simulation storage plane changes an information collection task in real time through monitoring an information collection item, and meanwhile, the simulation storage plane comprises a visualization module which is used for inquiring and retrieving historical node information stored in the simulation storage plane;
and the simulation decision plane receives the virtual network request of the simulation scene logic plane, generates a decision scheme, sends a decision result to the simulation control plane, and updates the resource state of the simulation data plane.
Based on the large-scale network simulation system, the bottom layer resources are dispatched and distributed, and the specific method comprises the following steps:
step S1, a logic plane of a simulation scene counts user service requests at a certain moment, and all service requests at a certain moment can be abstracted into a virtual network because corresponding underlying network nodes need to calculate resources and communication resources when responding to a certain service request. CPU resources required to be consumed by each node in the virtual network and bandwidth values of adjacent links represent underlying network computing resources and communication resources required by a certain service; sending the virtual network request to a simulation decision plane;
s2, the simulation control plane issues a control command to the simulation data plane according to the underlying network topology data provided by the simulation scene logic plane, instantiates the underlying network, and simultaneously the simulation storage plane updates the underlying network node state matrix M at the current moment n And underlying network link state matrix M p ；
S3, a simulation decision plane needs to combine a virtual network request test set and an underlying network model to train a virtual node mapping model; the virtual node mapping model designed by the invention is shown in figure 4, and adopts a reinforcement learning method to define the environmental state parameters of the underlying network nodes, the underlying network resource matrix and the reward function in turn, trains the reinforcement learning agent network by using a gradient descent method, then sends the virtual network request test set and the underlying network topology into the virtual node mapping model for training, and trains the reinforcement learning agent network according to the reward function value of each iteration; in particular, the amount of the solvent to be used,
s3.1, training the agent network by using a strategy gradient method, and adjusting a mapping strategy according to feedback of an environment, namely a reward function, wherein the higher the environment reward value is, the better the mapping scheme is; setting the current overall profit-to-cost ratio as a training reward function, and obtaining a group of complete virtual node mapping schemes in each iteration, wherein the aim is to maximize the long-term accumulated profit; the profit is the sum of the CPU values successfully mapped to the underlying network by the virtual node, and the cost is the underlying network resources consumed by the request for mapping the virtual node, namely:
wherein R is 1 Representing the above gain, the function CpuSum (n) v ) For calculating the CPU value, n, of a virtual node successfully arriving at the underlying node v Representing a respective virtual node; c 1 Representing the above cost, the function CpuCost (n) v ) Underlying network resources, n, for calculating consumption of a virtual node in a mapping process v Representing the corresponding virtual node.
The reward function is expressed as follows:
s3.2, selecting four parameters of computing resources, node link resources, node degree centrality and approximate centrality of each node to carry out node mapping strategy optimization of the proxy network; in particular, the amount of the solvent to be used,
(1) And computing resources: with C i To indicate the current available CPU resources of the ith node on the underlying network:
wherein, the function CPU () is used to calculate the remaining available CPU resources of the current node,
representing the corresponding underlying node.
(2) Node link resource: BW (l) is the bandwidth of link l of the ith node, and the bandwidths of all the adjacent links of node i are added together to use BW i Node link resources representing the ith node on the underlying network:
(3) Node degree centrality: by D i Reflecting the number of adjacent links owned by a node:
wherein
The degree of the ith node is, and m is the number of the nodes of the underlying network;
(4)、the approach to centrality: this attribute reflects the proximity of one node to other nodes, denoted J i Represents:
wherein d is ij Representing the number of edges on the shortest path between the node i and the node j;
after extracting the environmental state parameters of the nodes, representing the characteristic vector as V for the ith node of the underlying network i ＝(C i ,BW i ,D i ,J i ) T The node state matrix of the underlying network is denoted as M n ＝(V 1 ,V 2 ,…,V N ) T 。
S4, training the virtual link mapping model again on the basis of the completion of the training of the virtual node mapping model, as shown in FIG. 5; sending the virtual network request test set, the underlying network and the trained virtual node mapping model into a virtual link mapping model, and training a virtual link agent decision network by using a gradient descent method; in particular, the amount of the solvent to be used,
s4.1, training the agent network by using a strategy gradient method, and adjusting a mapping strategy according to feedback of an environment, namely a reward function, wherein the higher the environment reward value is, the better the mapping scheme is; setting the current overall profit-to-cost ratio as a training reward function, and obtaining a group of complete virtual node mapping schemes in each iteration, wherein the aim is to maximize the long-term accumulated profit; the profit is the sum of the link bandwidth values successfully mapped to the underlying network by the virtual network; the cost is the underlying network resources consumed by mapping the virtual network request, i.e.:
wherein R is 2 Representing the above benefit, function BWSum (l) v ) For calculating the bandwidth value, l, of a successful link to the underlying network v Representing respective virtual links；C 2 Representing the above cost, function BWCost (l) v ) Underlying network resources, hop (l), used to compute the consumption of a virtual link in the mapping process v ) Indicates the number of the bottom layer links occupied by a virtual link, l v Representing the corresponding virtual link.
The reward function is expressed as follows:
s4.2, when the virtual link is mapped to the bottom link, the link segmentation condition exists; the reachable path characteristics between each node pair in the underlying network are used as environment state parameters, and two parameters of bandwidth and medium centrality are selected for each underlying path to be optimized; in particular, the amount of the solvent to be used,
(1) Bandwidth: for all links between certain underlying paths, selecting the smallest link bandwidth value as the bandwidth of the physical path, that is:
wherein,
representing the kth underlay path of the underlay network.
(2) The center of the medium is as follows: for all links in the k-th underlying path, use d l Representing the number of shortest paths through link l, by D k Representing the number of all shortest paths through the kth underlying path, the intermediaries centrality of the kth underlying path is represented as:
wherein count represents the number of shortest path edges on path k;
after extracting the environmental state parameters of the underlying network path, the k-th itemUnderlying network paths, the feature vectors being represented as
The link state matrix of the underlying network is denoted as M p ＝(P 1 ,P 2 ,…,P N ) T 。
S5, extracting an environment state matrix of the simulation storage plane by the simulation decision plane, and sequentially performing node mapping and link mapping on the virtual network request by the reinforcement learning agent network of the simulation decision plane according to the latest environment state matrix; when the virtual network request cannot complete mapping, the simulation decision plane rejects the request and informs the user;
s6, when the simulation decision plane successfully answers a virtual network request, the decision result is sent to the simulation control plane, the simulation control plane issues a control command accordingly, the simulation data plane updates the resource state of the underlying network, and finally the updated underlying network node state matrix M is used for updating the resource state of the underlying network n And underlying network link state matrix M p And storing to the simulation storage plane.
For subsequently arriving virtual network requests, the mapping will be done in the steps described above.
The above description is only of the preferred embodiments of the present invention, and it should be noted that: it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention and these are intended to be within the scope of the invention.
Claims (2)
1. A large-scale network simulation system is characterized by comprising a simulation scene logic plane, a simulation control plane, a simulation data plane, a simulation storage plane and a simulation decision plane;
the simulation scene logic plane generates a corresponding underlying network model according to an actual scene, and the underlying network model is used for simulating a real network in the actual scene; the underlying network model comprises network nodes, links associated with the network nodes, network topology relations, software systems used by the network nodes and protocol stacks; the logic plane of the simulation scene counts the service requests of the users at a certain moment, and all the service requests at a certain moment can be abstracted into a virtual network because the corresponding calculation resources and communication resources of the underlying network nodes are needed for responding to a certain service request; the node values and the link values in the virtual network topology respectively represent underlying network computing resources and communication resources required by responding to corresponding service requests; the process of mapping the virtual network to the underlying network is the process of scheduling and distributing underlying network resources; specifically, the following are shown:
G V →G S
the virtual network is denoted G V ＝(N V ,L V ),N V Representing virtual network nodes, L V Representing a virtual network link; the underlying network is denoted G S ＝(N S ,L S )，N S Indicating underlying network nodes, L S Representing an underlying network path;
the simulation control plane comprises an SDN controller and a node controller; the SDN controller is used for maintaining a topological structure of a simulation network in a simulation data plane and controlling topological linkage of the simulation network by dynamically updating flow table contents of an SDN switch; the node controller is used for issuing a service simulation instruction and monitoring the resource attribute of each node; the node controller dynamically switches the simulation service types of the simulation network nodes according to the decision result of the simulation decision plane;
the simulation data plane constructs the same number of simulation nodes according to the network model data, and the resource attributes of the simulation nodes, the software system and the protocol stack used by the simulation nodes are consistent with the network model and are used for instantiating the network model generated in the simulation scene logic plane and simulating the real network nodes; the simulation data plane simulates wireless links among nodes according to network model data, and the uplink and downlink bandwidths, the error rate and the packet loss rate of the links among the nodes are consistent with those of a network model; the simulation data plane supports communication between the virtual nodes and the physical nodes;
the simulation storage plane captures and stores node flow information, network situation information and resource occupation information generated by the simulation data plane in the simulation process, and transmits the node flow information, the network situation information and the resource occupation information to the simulation decision plane for decision making and integration links; the simulation storage plane changes an information collection task in real time through monitoring an information collection item, and meanwhile, the simulation storage plane comprises a visualization module which is used for inquiring and retrieving historical node information stored in the simulation storage plane;
and the simulation decision plane receives the virtual network request of the simulation scene logic plane, generates a decision scheme, sends a decision result to the simulation control plane, and updates the resource state of the simulation data plane.
2. A method for dynamically scheduling and allocating underlying network resources by using the large-scale network simulation system according to claim 1, comprising the steps of:
s1, a logic plane of a simulation scene counts user service requests at a certain moment, and all service requests at a certain moment can be abstracted into a virtual network because corresponding underlying network node computing resources and communication resources are needed for responding to a certain service request; CPU resources required to be consumed by each node in the virtual network and bandwidth values of adjacent links represent underlying network computing resources and communication resources required by a certain service; sending the virtual network request to a simulation decision plane;
s2, the simulation control plane issues a control command to the simulation data plane according to the underlying network topology data provided by the simulation scene logic plane, instantiates the underlying network, and simultaneously the simulation storage plane updates the underlying network node state matrix M at the current moment n And underlying network link state matrix M p ；
S3, the simulation decision plane needs to combine a virtual network request test set and an underlying network model to train a virtual node mapping model; sequentially defining environment state parameters of the underlying network nodes, an underlying network resource matrix and a reward function by adopting a reinforcement learning method, training a reinforcement learning agent network by using a gradient descent method, then sending a virtual network request test set and underlying network topology into a virtual node mapping model for training, and training the reinforcement learning agent network according to the reward function value of each iteration; the training of the virtual node mapping model specifically comprises:
s3.1, training the agent network by using a strategy gradient method, and adjusting a mapping strategy according to feedback of an environment, namely a reward function, wherein the higher the environment reward value is, the better the mapping scheme is; setting the current overall profit-to-cost ratio as a training reward function, and obtaining a group of complete virtual node mapping schemes in each iteration, wherein the aim is to maximize the long-term accumulated profit; the yield is the sum of the CPU values successfully mapped to the underlying network by the virtual nodes, and the cost is the underlying network resources consumed by mapping the virtual network request, namely:
wherein R is 1 Representing the above gain, the function CpuSum (n) v ) For calculating the CPU value, n, of a virtual node successfully mapped onto an underlying node v Representing the respective virtual node; cost 1 Representing the above cost, the function CpuCost (n) v ) Underlying network resources, n, consumed by a virtual node in a computational mapping process v Representing a respective virtual node;
the reward function is expressed as follows:
s3.2, selecting four parameters of computing resources, node link resources, node degree centrality and approximate centrality of each node to carry out node mapping strategy optimization of the proxy network; in particular, the amount of the solvent to be used,
(1) And computing resources: with C i To indicate the current available CPU resources of the ith node on the underlying network:
wherein the function CPU () is used to calculate the remaining available CPU resources of the current node,
representing the corresponding underlying node;
(2) Node link resources: BW (l) is the bandwidth of link l of the ith node, and the bandwidths of all adjacent links of node i are added together using BW i Node link resources representing the ith node on the underlying network:
(3) Node degree centrality: by D i Reflecting the number of adjacent links owned by a node:
wherein
The degree of the ith node is, and m is the number of the nodes of the underlying network;
(4) And the centering property: this attribute reflects the proximity of one node to other nodes, denoted J i Represents:
wherein d is ij Representing the number of edges on the shortest path between the node i and the node j;
after extracting the environmental state parameters of the nodes, representing the characteristic vector as V for the ith node of the underlying network i ＝(C i ,BW i ,D i ,J i ) T The node state matrix of the underlying network is denoted as M n ＝(V 1 ,V 2 ,…,V N ) T ；
S4, training the virtual link mapping model again on the basis of finishing the training of the virtual node mapping model; sending the virtual network request test set, the underlying network and the trained virtual node mapping model into a virtual link mapping model, and training a virtual link agent decision network by using a gradient descent method; the training of the virtual link mapping model specifically includes:
s4.1, training the agent network by using a strategy gradient method, and adjusting a mapping strategy according to feedback of an environment, namely an incentive function, wherein the higher the environment incentive value is, the better the mapping scheme is; setting the current overall profit-to-cost ratio as a training reward function, and obtaining a group of complete virtual node mapping schemes in each iteration, wherein the aim is to maximize the long-term accumulated profit; the profit is the sum of the link bandwidth values successfully mapped to the underlying network by the virtual network; the cost is the underlying network resources consumed by mapping the virtual network request, i.e.:
wherein R is 2 Representing the above benefit, function BWSum (l) v ) Used for calculating the link bandwidth value l of a virtual link successfully mapped to the underlying network v Representing the corresponding virtual link; cost 2 Representing the above cost, function BWCost (l) v ) Underlying network resources, hop (l), used to compute the consumption of a virtual link in the mapping process v ) Indicates the number of the bottom layer links occupied by a virtual link, l v Representing the corresponding virtual link;
the reward function is expressed as follows:
s4.2, when the virtual link is mapped to the bottom link, the link segmentation condition exists; the reachable path characteristics between each node pair in the underlying network are used as environment state parameters, and two parameters of bandwidth and medium centrality are selected for each underlying path to be optimized; in particular, the amount of the solvent to be used,
(1) Bandwidth: for all links between certain underlying paths, selecting the smallest link bandwidth value as the bandwidth of the physical path, that is:
wherein,
a kth underlay path representing an underlay network;
(2) The center of the medium is as follows: for all links in the k-th underlying path, use d l Representing the number of shortest paths through link l, by D k Representing the number of all shortest paths through the kth underlying path, the intermediaries centrality of the kth underlying path is represented as:
wherein count represents the number of shortest path edges on path k;
after extracting the environmental state parameters of the underlying network path, expressing the characteristic vector of the k-th underlying network path as
The link state matrix of the underlying network is denoted as M p ＝(P 1 ,P 2 ,…,P N ) T ；
S5, extracting an environment state matrix of the simulation storage plane by the simulation decision plane, and sequentially performing node mapping and link mapping on the virtual network request by the reinforcement learning agent network of the simulation decision plane according to the latest environment state matrix; when the virtual network request cannot complete mapping, the simulation decision plane rejects the request and informs the user;
s6, when the simulation decision plane successfully answers a virtual network request, the decision result is sent to the simulation control plane, the simulation control plane issues a control command according to the decision result, the simulation data plane updates the resource state of the underlying network, and finally the updated underlying network node state matrix M is used for updating the resource state of the underlying network n And underlying network link state matrix M p And storing to the simulation storage plane.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111314346.5A CN114050961B (en) | 2021-11-08 | 2021-11-08 | Large-scale network simulation system and resource dynamic scheduling and distributing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111314346.5A CN114050961B (en) | 2021-11-08 | 2021-11-08 | Large-scale network simulation system and resource dynamic scheduling and distributing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114050961A CN114050961A (en) | 2022-02-15 |
| CN114050961B true CN114050961B (en) | 2022-10-18 |
Family
ID=80207489
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111314346.5A Active CN114050961B (en) | 2021-11-08 | 2021-11-08 | Large-scale network simulation system and resource dynamic scheduling and distributing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114050961B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114615183B (en) * | 2022-03-14 | 2023-09-05 | 广东技术师范大学 | Routing method, device, computer equipment and storage medium based on resource prediction |
| CN117693059A (en) * | 2022-08-30 | 2024-03-12 | 中兴通讯股份有限公司 | Resource scheduling method, device and system, storage medium and electronic system |
| CN115729714A (en) * | 2023-01-06 | 2023-03-03 | 之江实验室 | Resource allocation method, device, storage medium and electronic equipment |
| CN117293807B (en) * | 2023-09-25 | 2024-07-26 | 上海能优网电力科技有限公司 | Multi-time scale optimization method and system for information side model of power distribution network |
| CN117714306B (en) * | 2024-02-06 | 2024-04-16 | 湖南天冠电子信息技术有限公司 | Ethernet performance optimization method based on scheduling algorithm |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102223281A (en) * | 2011-06-24 | 2011-10-19 | 清华大学 | Method for remapping resource demand dynamic change based on virtual network |
| CN109889930A (en) * | 2019-03-26 | 2019-06-14 | 重庆邮电大学 | A kind of virtual optical network mapping method and device for combining energy consumption and load balancing |
| CN110365514A (en) * | 2019-05-24 | 2019-10-22 | 北京邮电大学 | SDN multistage mapping method of virtual network and device based on intensified learning |
| CN113193999A (en) * | 2021-04-29 | 2021-07-30 | 东北大学 | Virtual network mapping method based on depth certainty strategy gradient |
| CN113328938A (en) * | 2021-05-25 | 2021-08-31 | 电子科技大学 | Network autonomous intelligent management and control method based on deep reinforcement learning |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11153229B2 (en) * | 2018-01-19 | 2021-10-19 | Ciena Corporation | Autonomic resource partitions for adaptive networks |
| CN112651130A (en) * | 2020-12-28 | 2021-04-13 | 航天科工网络信息发展有限公司 | Decision support-oriented virtual-real mapping parallel simulation system |
-
2021
- 2021-11-08 CN CN202111314346.5A patent/CN114050961B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102223281A (en) * | 2011-06-24 | 2011-10-19 | 清华大学 | Method for remapping resource demand dynamic change based on virtual network |
| CN109889930A (en) * | 2019-03-26 | 2019-06-14 | 重庆邮电大学 | A kind of virtual optical network mapping method and device for combining energy consumption and load balancing |
| CN110365514A (en) * | 2019-05-24 | 2019-10-22 | 北京邮电大学 | SDN multistage mapping method of virtual network and device based on intensified learning |
| CN113193999A (en) * | 2021-04-29 | 2021-07-30 | 东北大学 | Virtual network mapping method based on depth certainty strategy gradient |
| CN113328938A (en) * | 2021-05-25 | 2021-08-31 | 电子科技大学 | Network autonomous intelligent management and control method based on deep reinforcement learning |
Non-Patent Citations (2)
| Title |
|---|
| A Dynamic and Collaborative Multi-Layer Virtual Network Embedding Algorithm in SDN Based on Reinforcement Learning;M. Lu, Y. Gu and D. Xie;《IEEE Transactions on Network and Service Management》;20200729;全文 * |
| 基于数据驱动的虚拟网络映射算法;陈旭;《中国优秀硕士学位论文全文数据库（电子期刊）》;20181115(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114050961A (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114050961B (en) | Large-scale network simulation system and resource dynamic scheduling and distributing method | |
| CN109714219A (en) | A kind of virtual network function fast mapping algorithm based on satellite network | |
| CN113364850B (en) | Software-defined cloud-edge collaborative network energy consumption optimization method and system | |
| CN113794494B (en) | Edge computing system and computing unloading optimization method for low-orbit satellite network | |
| CN105515987B (en) | A kind of mapping method based on SDN framework Virtual optical-fiber networks | |
| CN108566659A (en) | A kind of online mapping method of 5G networks slice based on reliability | |
| CN108874525A (en) | A kind of service request distribution method towards edge calculations environment | |
| CN108111335B (en) | A kind of method and system of scheduling and link virtual network function | |
| CN113660681B (en) | Multi-agent resource optimization method applied to unmanned aerial vehicle cluster auxiliary transmission | |
| Lei et al. | Congestion control in SDN-based networks via multi-task deep reinforcement learning | |
| Guan et al. | An intelligent wireless channel allocation in HAPS 5G communication system based on reinforcement learning | |
| CN115580882A (en) | Dynamic network slice resource allocation method and device, storage medium and electronic equipment | |
| CN112199154B (en) | Reinforced learning training system and method based on distributed collaborative sampling center type optimization | |
| Bouzidi et al. | Dynamic clustering of software defined network switches and controller placement using deep reinforcement learning | |
| CN116489708B (en) | Meta universe oriented cloud edge end collaborative mobile edge computing task unloading method | |
| CN110247795A (en) | A kind of cloud net resource service chain method of combination and system based on intention | |
| CN116893861A (en) | Multi-agent cooperative dependency task unloading method based on space-ground cooperative edge calculation | |
| CN111343095B (en) | Method for realizing controller load balance in software defined network | |
| Na et al. | An evolutionary game approach on IoT service selection for balancing device energy consumption | |
| Hu et al. | Dynamic task offloading in MEC-enabled IoT networks: A hybrid DDPG-D3QN approach | |
| CN112073237A (en) | Large-scale target network construction method in cloud edge architecture | |
| EP4024212A1 (en) | Method for scheduling interference workloads on edge network resources | |
| CN116755867B (en) | Satellite cloud-oriented computing resource scheduling system, method and storage medium | |
| Zhu et al. | Deep reinforcement learning-based edge computing offloading algorithm for software-defined IoT | |
| CN115225512B (en) | Multi-domain service chain active reconfiguration mechanism based on node load prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |