CN115834371A - Space-ground converged network cross-domain SFC deployment method based on hybrid state synchronous DRL - Google Patents

Space-ground converged network cross-domain SFC deployment method based on hybrid state synchronous DRL Download PDF

Info

Publication number
CN115834371A
CN115834371A CN202211457628.5A CN202211457628A CN115834371A CN 115834371 A CN115834371 A CN 115834371A CN 202211457628 A CN202211457628 A CN 202211457628A CN 115834371 A CN115834371 A CN 115834371A
Authority
CN
China
Prior art keywords
network
actor
domain
deployment
deployed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202211457628.5A
Other languages
Chinese (zh)
Other versions
CN115834371B (en
Inventor
武楠
李浩阳
张婷婷
李彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN202211457628.5A priority Critical patent/CN115834371B/en
Publication of CN115834371A publication Critical patent/CN115834371A/en
Application granted granted Critical
Publication of CN115834371B publication Critical patent/CN115834371B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention discloses a cross-domain SFC deployment method of a space-ground fusion network based on hybrid state synchronization DRL (data logging language), which can effectively reduce huge expenses caused by real-time synchronization of the global network state of the space-ground fusion network through key parameter synchronization and is suitable for a satellite control node with limited resources. On the other hand, by virtue of the synchronization of the state information of the global network and the idea of digital twin, the virtual copy Actor-Critic network of the A3C has rich and sufficiently real sample sets, and the success rate of SFC deployment of the global Actor-Critic network and the real copy Actor-Critic network can be further improved; meanwhile, the actual copy Actor-Critic network of the A3C makes an optimal decision on cross-domain SFC deployment of the world fusion network based on real-time synchronous network key state parameters, and supports world fusion network services.

Description

Space-ground converged network cross-domain SFC deployment method based on hybrid state synchronous DRL
Technical Field
The invention belongs to the technical field of network control, and particularly relates to a space-ground convergence network cross-domain SFC deployment method based on hybrid state synchronous DRL.
Background
The heaven and earth converged network is formed by fusing a satellite and earth network by relying on a ground network and expanding a satellite communication network, has the advantages of high bearing capacity, wide coverage range, flexible networking mode and the like, and is a key technology of the next generation of information network. However, the characteristics of large space-time scale, strong topology dynamics, heterogeneous links and the like of the space-ground converged network bring huge challenges to the network service support capability.
Deep Reinforcement Learning (DRL) is an effective decision optimization method, and has wide application in the field of network control. By introducing the intelligent plane, the DRL carries out iterative optimization on self decision based on the perceived network state and issues the decision to the control plane, thereby realizing the optimal control of the network. In particular, the cross-domain service function chain deployment (SFC) of the world convergence network can be modeled as a Markov Decision Process (MDP), and the Bellman equation is iteratively solved by the DRL to support the world convergence network service. However, the traditional DRL method faces the problems of high network state space dimension, high data set acquisition overhead, difficult cross-domain collaboration and the like.
In order to realize the optimal deployment of the SFC, the intelligent plane needs to acquire the global network state information in real time, and the complexity of the DRL algorithm is increased while huge communication overhead is brought to the world convergence network. On the other hand, in a world-wide converged network with limited resources, the acquisition of network state data is limited, so that the sample set is small in scale and difficult to support the training of the DRL.
Disclosure of Invention
In view of this, the invention provides a space-ground convergence network cross-domain SFC deployment method based on a hybrid state synchronous DRL based on an asynchronous dominant Actor-critical algorithm (A3C) in deep reinforcement learning. The A3C has a global Actor-criticic network and a plurality of copy Actor-criticic networks, and can asynchronously and parallelly realize the updating of the neural network. Specifically, the invention designs a hybrid network state synchronization mechanism, which is used for synchronizing key parameters of network states of all domains of a world fusion network in real time, training a partial replica Actor-critical network and a global Actor-critical network, synchronizing global network state information of the world fusion network when the network is idle, generating the key parameters through digital twinning, and training other replica Actor-critical networks and global Actor-critical networks, thereby realizing cross-domain SFC deployment.
A space-ground fusion network cross-domain SFC deployment method based on hybrid state synchronous DRL adopts an A3C network for training, wherein: one Actor-critical network is used as a global Actor-critical network theta G P Actor-critical networks form a real copy Actor-critical network set
Figure BDA0003953710870000021
Q Actor-critical networks form a virtual copy Actor-critical network set
Figure BDA0003953710870000022
P and Q are not less than 1;
determining a required heaven-earth converged network key parameter vector when deploying the ith virtual network function VNF of the heaven-earth converged network
Figure BDA0003953710870000023
And networking network state vector s of the world converged network i ；
For training optimization of the A3C network, the following two processes are carried out simultaneously:
the first process is as follows: for reality copy Actor-criticc network
Figure BDA0003953710870000024
In the i-th virtual network function VNF v i When deployment is carried out, key parameters of the network are fused according to the real-time synchronization of the heaven and the earth
Figure BDA0003953710870000025
To obtain s i Input of
Figure BDA0003953710870000026
Gets v i Deployment decision a i And sends it down to the control plane; control ofPlane according to a i V is to be i Deployment to respective domains according to v i Get a reward based on the deployed results of (a) i Calculating a policy gradient, updating in sequence
Figure BDA0003953710870000027
The Critic network and the Actor network are used until all VNFs are deployed; when the SFC is deployed once, the deployment is updated
Figure BDA0003953710870000028
The gradient of (a) is uploaded to a global Actor-critical network Θ G And update the theta sequentially G Finally copying the parameters of the Actor network and the critical network to a real copy Actor-critical network
Figure BDA0003953710870000029
The second process is as follows: for a virtual copy Actor-critical network, the network is
Figure BDA00039537108700000210
In the i-th virtual network function VNF v i When deployment is carried out, key parameters are calculated according to the global network state of intelligent plane synchronization
Figure BDA00039537108700000211
And obtaining s i Input of
Figure BDA00039537108700000212
Gets v i Deployment decision a i In the world-wide integration network simulation software, a is executed based on the network state of each domain i V is to be i Deployment to corresponding Domain, v i Simulating the execution situation in the domain to obtain the return, and obtaining the return based on the return and a i Calculating a policy gradient, updating in sequence
Figure BDA00039537108700000213
The Critic network and the Actor network until all VNFs are deployed; when it is finishedAfter one SFC deployment, updating
Figure BDA00039537108700000214
The gradient of (a) is uploaded to a global Actor-critical network Θ G And update the theta sequentially G Finally copying the parameters of the Actor network and the critical network to a real copy Actor-critical network
Figure BDA00039537108700000215
And completing the deployment of the SFC according to the first process and the second process.
Preferably, the world fusion network key parameter vector
Figure BDA00039537108700000216
Comprises the following steps:
Figure BDA00039537108700000217
networking network state vector s of the world convergence network i Comprises the following steps:
Figure BDA00039537108700000218
；
the method comprises the following steps that a set of N network domains of a world fusion network is set to be D, a set of K inter-domain links is set to be L, a set of I virtual network functions VNF contained in the SFC is set to be V, and a set of J resource types is set to be R; the interval 1. Ltoreq. I'. Ltoreq.I comprises I, the ith VNF v i An authorized domain that e V can be deployed is
Figure BDA0003953710870000031
For j resource type r j E.g. the requirement of R is alpha i,j The bandwidth required for cross-domain transmission is w i Maximum time delay of the whole SFC is tau + (ii) a N-th field d n E.g. D resource r j The balance of
Figure BDA0003953710870000032
It is to VNF v i Has a processing delay of
Figure BDA0003953710870000033
The kth inter-domain link l k The bandwidth of E L is gamma k Time delay of τ l,k (ii) a Wherein when d n When it is an unauthorized domain
Figure BDA0003953710870000034
Gamma when no link is present k ＝0，τ l,k ＝∞；
{χ i' } 1≤i'≤I For indicating the state of the currently deployed VNF, if ^ x i' If n is not zero, then v is i' Deployed in domain d n Upper, if X i' If =0, then v is represented i' Not yet deployed, and has χ i' ＝0,i'≥i。
Preferably, when the A3C network is trained, in the Actor network, the set constraint condition is implemented by using an action mask, that is, a decision that does not satisfy the constraint is shielded; wherein the constraint condition comprises:
1) One VNF can only be deployed into one domain, and a physical link must exist between two consecutively deployed VNFs;
2) Resources occupied by all Virtual Network Functions (VNFs) deployed in one domain cannot exceed the resource margin of the domain;
3) The bandwidth occupied by all the Virtual Network Functions (VNFs) carried by one link cannot exceed the bandwidth of the link;
4) The sum of the processing delay and the cross-domain transmission delay in all VNF domains cannot exceed the total delay.
Preferably, the return is calculated by a return function as follows:
Figure BDA0003953710870000035
wherein, S ∈ {0,1} is a Boolean variable, which indicates whether the service function chain is successfully deployed, S =1 successfully, and S =0 unsuccessfully; r S And R F Reward punishment of success and failure of deployment respectively；
Figure BDA0003953710870000036
Representing the resources within the domain required for deploying all virtual network functions VNF V in constraint 2);
c l,k representing the link bandwidth required for deploying all virtual network functions VNF V in constraint 3);
τ v,l representing the processing delay required for deploying all virtual network functions VNF V in constraint 4);
Figure BDA0003953710870000037
p l,k ，p v,l are respectively as
Figure BDA0003953710870000038
c l,k And τ v,l The corresponding weight factor.
The invention has the following beneficial effects:
in the cross-domain SFC deployment method of the heaven and earth fusion network based on the hybrid state synchronization DRL, on one hand, huge expenses caused by real-time synchronization of the global network state of the heaven and earth fusion network can be effectively reduced through key parameter synchronization, and the cross-domain SFC deployment method is suitable for satellite control nodes with limited resources. On the other hand, by virtue of the synchronization of the state information of the global network and the idea of digital twin, the virtual copy Actor-Critic network of the A3C has rich and sufficiently real sample sets, and the success rate of SFC deployment of the global Actor-Critic network and the real copy Actor-Critic network can be further improved. Meanwhile, the actual copy Actor-Critic network of the A3C makes an optimal decision on cross-domain SFC deployment of the world fusion network based on real-time synchronous network key state parameters, and supports world fusion network services.
Drawings
Fig. 1 is a schematic block diagram of a space-ground convergence network cross-domain SFC deployment based on hybrid state synchronous DRL.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The invention aims to overcome the defects of the prior art and solve the problem of cross-domain SFC deployment under space-time large scale of a space-ground converged network, and provides a space-ground converged network cross-domain SFC deployment method based on hybrid state synchronous DRL. Compared with the SFC deployment method based on the global state synchronization and the key state synchronization, the method provided by the invention has the advantages of low synchronization overhead, high deployment success rate, sufficient training samples and the like.
The method is realized by the following technical scheme:
aiming at cross-domain service function chain deployment under large space-time scale, the invention introduces a deep reinforcement learning technology into a heaven-earth fusion network, decouples the heaven-earth fusion network from the aspects of logic and function, and constructs a logic network model based on a physical plane, a control plane and an intelligent plane, as shown in figure 1.
In the method provided by the invention, the real-time synchronous key parameter is used for updating the real copy Actor-criticic network in the A3C, and the synchronous global network state generates the virtual key parameter by a digital twin method and is used for updating the virtual copy Actor-criticic network. When the real copy Actor-critical network or the virtual copy Actor-critical network completes a round of updating, the strategy gradient of the real copy Actor-critical network or the virtual copy Actor-critical network is used for updating the global Actor-critical network. Let the global Actor-Critic network be Θ G There are P real copies of the Actor-Critic network as
Figure BDA0003953710870000041
There are Q virtual copy Actor-criticic networks as
Figure BDA0003953710870000042
For the network state, a set of N network domains of the world convergence network is D, a set of K inter-domain links is L, a set of I Virtual Network Functions (VNFs) contained in the SFC is V, a set of J resource types is R, and the ith VNfv is i An authorized domain that e V can be deployed is
Figure BDA0003953710870000043
For j resource type r j E.g. requirement of R is alpha i,j The bandwidth required for cross-domain transmission is w i The maximum time delay of the whole SFC is tau + . N-th field d n E.g. D resource r j The balance of
Figure BDA0003953710870000044
(if d) n Is an unauthorized domain, then
Figure BDA0003953710870000051
) Of VNfv i Has a processing delay of
Figure BDA0003953710870000052
The kth inter-domain link l k The bandwidth of E L is gamma k Time delay of τ l,k (in the absence of a link γ) k ＝0，τ l,k = ∞). Thus, VNfv is being deployed i The key network parameter Delta needed for neural network training i Network topology, authorized domain, resource allowance of each domain, processing time delay, bandwidth and time delay of inter-domain link, namely:
Figure BDA0003953710870000053
wherein
Figure BDA0003953710870000054
Is v is i Parameters obtained after deployment, the values of which are related to the VNF deployment algorithm used in the domain, are used
Figure BDA0003953710870000055
Indicating the ability to obtain directly (except for
Figure BDA0003953710870000056
External) to the network. Thus, deployment VNFv i The temporal network state may be represented as a vector
Figure BDA0003953710870000057
Wherein, the interval 1 is not less than I' not more than I comprises I; { X i' } 1≤i'≤I For indicating the state of the currently deployed VNF, if ^ x i' If n is not zero, then v is i' Deployed in domain d n Upper, if X i' If =0, then v is represented i' Not yet deployed and has a chi i' ＝0,i'≥i。
For network decision, VNfv i Deployable decision is a i ∈D i . It is assumed that the shortest path from each domain to the other domains is known (which can be solved by Dijkstra). Suppose that the value of the Boolean variable b epsilon {0,1} is 0 indicates that the event is false (NO), and the value of 1 indicates that the event is true (YES).
1) Using Boolean variables, taking into account the gamut mapping constraints
Figure BDA0003953710870000058
Denotes v i Whether or not to be deployed to domain d n The above. There is therefore a gamut mapping constraint, namely:
Figure BDA0003953710870000059
the constraint indicates that a VNF can only be deployed into one domain.
In addition, two successive VNFvs i And v i+1 The following cross-domain mapping constraints must be satisfied:
Figure BDA00039537108700000510
where the Boolean variable ρ n,m E {0,1} represents the field d n To domain d m Whether or not a link exists, the constraint indicating that a physical link must exist between two VNFs deployed in succession. For example, when domain dn to domain d m When there is no link, the link is not present,
Figure BDA00039537108700000511
denotes v i And v i+1 Cannot be deployed to domain d, respectively n And domain d m The above.
2) Considering the intra-domain resource constraints, for a successfully deployed SFC (i.e., all VNFVs), it will be directed to the intra-domain resources r j Is satisfied with
Figure BDA0003953710870000061
The constraint indicates that the resources occupied by all VNFs deployed within a domain cannot exceed the domain resource margin.
3) Considering the cross-domain link bandwidth constraint, for a successfully deployed SFC (i.e., all VNF V), the bandwidth constraint is
Figure BDA0003953710870000062
Wherein the Boolean variable ξ n,m,k E {0,1} represents d n To domain d m Whether or not the path of (1) contains l k The constraint means that the bandwidth occupied by all VNFs carried by a link cannot exceed the link bandwidth.
4) Considering the cross-domain link latency constraint, for a successfully deployed SFC (i.e., all VNFVs), the latency constraint is:
Figure BDA0003953710870000063
the constraint represents the total delay of one SFC, i.e. the sum of the processing delay in all VNF domains and the cross-domain transmission delay, cannot exceed the total delay.
In the Actor network, the constraint condition is realized by adopting an action mask, namely, the decision which does not meet the constraint is shielded.
For the return function, a weight factor is defined
Figure BDA0003953710870000064
p l,k ，p v,l Representing the cost overhead of a unit resource (computation, storage, bandwidth, time delay, etc.) to the heaven-earth converged network, the reward function is:
Figure BDA0003953710870000065
wherein the Boolean variable S is equal to {0,1} and represents whether the service function chain is deployed successfully, R S And R F Respectively, successful deployment and failed deployment.
For training optimization of the A3C network, two cases will be distinguished.
First, for a real copy Actor-critical network
Figure BDA0003953710870000066
In the ith VNfv to SFC i When deployment is carried out, key parameters of the network are fused according to the real-time synchronization of the heaven and the earth
Figure BDA0003953710870000067
To obtain s i Input of
Figure BDA0003953710870000068
Gets v i Deployment decision a i And sent to the control plane. Control plane according to a i V is to be i Deployment to respective domains according to v i Post-deployment result derived parameters
Figure BDA0003953710870000069
And calculating a reward R based on R and a i Calculating a policy gradient, updating in sequence
Figure BDA00039537108700000610
The Critic network and the Actor network until all VNFVs of the SFC are deployed. When SFC is completed once, updating
Figure BDA00039537108700000611
The gradient of (a) is uploaded to a global Actor-critical network Θ G And update the theta sequentially G Finally copying the parameters of the Actor network and the critical network to a real copy Actor-critical network
Figure BDA00039537108700000612
Second, for a virtual copy Actor-critical network, the network is
Figure BDA0003953710870000071
In the ith VNfv to SFC i When deployment is carried out, key parameters are calculated according to the global network state of intelligent plane synchronization
Figure BDA0003953710870000072
And obtaining s i Input of
Figure BDA0003953710870000073
Gets v i Deployment decision a i With the aid of the idea of digital twin, in network simulation software, a is executed based on the network state of each domain i V is to be i Deploying to the corresponding domain, and randomly running the existing intra-domain VNF embedding algorithm to v i Simulating the execution condition in the domain to obtain the parameters
Figure BDA0003953710870000074
And calculating a reward R based on R and a i Calculating a policy gradient, updating in sequence
Figure BDA0003953710870000075
And (4) the Critic network and the Actor network until all VNF V of the SFC are deployed. When SFC is completed once, updating
Figure BDA0003953710870000076
The gradient of (a) is uploaded to a global Actor-critical network Θ G And update the theta sequentially G Finally copying the parameters of the Actor network and the criticic network to a real copy Actor-criticic networkIs composed of
Figure BDA0003953710870000077
The training of the two networks is carried out in parallel without mutual interference.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (4)

1. A space-ground fusion network cross-domain SFC deployment method based on hybrid state synchronous DRL is characterized in that an A3C network is adopted for training, wherein: one Actor-critical network is used as a global Actor-critical network theta G P Actor-critical networks form a real copy Actor-critical network set
Figure FDA0003953710860000011
Q Actor-critical networks form a virtual copy Actor-critical network
Figure FDA0003953710860000012
P and Q are not less than 1;
determining a required heaven-earth converged network key parameter vector when deploying the ith virtual network function VNF of the heaven-earth converged network
Figure FDA0003953710860000013
And networking network state vector s of the world converged network i ；
For training optimization of the A3C network, the following two processes are carried out simultaneously:
the first process is as follows: for real copy Actor-Critic network
Figure FDA0003953710860000014
In the i-th virtual network function VNF v i When deployment is carried out, the network is converged according to the real-time synchronous heaven and earthKey parameter
Figure FDA0003953710860000015
To obtain s i Input of
Figure FDA0003953710860000016
Gets v i Deployment decision a i And sends it to the control plane; control plane according to a i V is to be i Deployment to respective domains according to v i Get a reward based on the deployed results of (a) i Calculating a policy gradient, updating in sequence
Figure FDA0003953710860000017
The Critic network and the Actor network until all the VNFs are deployed; when one SFC deployment is finished, updating
Figure FDA0003953710860000018
The gradient of (a) is uploaded to a global Actor-critical network Θ G And update the theta sequentially G Finally copying the parameters of the Actor network and the critical network to a real copy Actor-critical network
Figure FDA0003953710860000019
The second process is as follows: for a virtual copy Actor-critical network, the network is
Figure FDA00039537108600000110
In the i-th virtual network function VNF v i When deployment is carried out, key parameters are calculated according to the global network state of intelligent plane synchronization
Figure FDA00039537108600000111
And obtain s i Inputting of
Figure FDA00039537108600000112
Gets v i Deployment decision a i In the world-wide integration network simulation software, a is executed based on the network state of each domain i V is to be i Deployment to corresponding Domain, v i Simulating the execution situation in the domain to obtain the return, and obtaining the return based on the return and a i Calculating a policy gradient, updating in sequence
Figure FDA00039537108600000113
The Critic network and the Actor network until all VNFs are deployed; when one deployment is completed, updating
Figure FDA00039537108600000114
The gradient of (a) is uploaded to a global Actor-Critic network Θ G And update the theta sequentially G Finally copying the parameters of the Actor network and the critical network to a real copy Actor-critical network
Figure FDA00039537108600000115
The deployment of SFCs (i.e., all VNFVs) is accomplished in accordance with the first procedure and the second procedure.
2. The method of claim 1, wherein the space-ground converged network cross-domain SFC deployment method based on hybrid state synchronous DRL is characterized in that the space-ground converged network key parameter vector
Figure FDA00039537108600000116
Comprises the following steps:
Figure FDA0003953710860000021
networking network state vector s of the world convergence network i Comprises the following steps:
Figure FDA0003953710860000028
the method comprises the following steps that a set of N network domains of a world convergence network is set to be D, a set of K inter-domain links is set to be L, a set of I virtual network functions VNF contained in an SFC is set to be V, and a set of J resource types is set to be R; the interval 1. Ltoreq. I'. Ltoreq.I comprises I, the ith VNF v i An authorized domain that e V can be deployed is
Figure FDA0003953710860000022
For j resource type r j E.g. the requirement of R is alpha i,j The bandwidth required for cross-domain transmission is w i Maximum time delay of the whole SFC is tau + (ii) a N-th field d n E.g. D resource r j The balance of
Figure FDA0003953710860000023
It is to VNF v i Has a processing delay of
Figure FDA0003953710860000024
The kth inter-domain link l k The bandwidth of E L is gamma k Time delay of τ l,k (ii) a Wherein when d n When it is an unauthorized domain
Figure FDA0003953710860000025
Gamma when no link is present k ＝0，τ l,k ＝∞；
{χ i' } 1≤i'≤I For indicating the state of the currently deployed VNF, if ^ x i' If n is not zero, then v is i' Deployed in domain d n Upper, if X i' If =0, then v is represented i' Not yet deployed, and has χ i' ＝0,i'≥i。
3. The method for deploying SFC across heaven and earth fusion network based on hybrid state synchronous DRL according to claim 2, wherein when training the A3C network, in the Actor network, the action mask is adopted to realize the set constraint condition, i.e. the decision not meeting the constraint is shielded; wherein the constraint condition comprises:
1) One VNF can only be deployed into one domain, and a physical link must exist between two consecutively deployed VNFs;
2) Resources occupied by all Virtual Network Functions (VNFs) deployed in one domain cannot exceed the resource margin of the domain;
3) The bandwidth occupied by all the Virtual Network Functions (VNFs) carried by one link cannot exceed the bandwidth of the link;
4) The sum of the processing delay and the cross-domain transmission delay in all VNF domains cannot exceed the total delay.
4. The method for deploying SFCs across the heaven and earth converged network based on hybrid state synchronous DRL of claim 3, wherein the reward is calculated by a reward function as follows:
Figure FDA0003953710860000026
wherein, S ∈ {0,1} is a Boolean variable, which indicates whether the service function chain is successfully deployed, S =1 successfully, and S =0 unsuccessfully; r S And R F Respectively being reward punishment of successful deployment and failed deployment;
Figure FDA0003953710860000027
representing the resources within the domain required by all the VNFVs of the virtual network function in constraint 2);
c l,k representing the link bandwidth required for deploying all virtual network functions VNF V in constraint 3);
τ v,l representing the processing delay required for deploying all virtual network functions VNF V in constraint 4);
Figure FDA0003953710860000031
p l,k ，p v,l are respectively as
Figure FDA0003953710860000032
c l,k And τ v,l The corresponding weight factor.
CN202211457628.5A 2022-11-21 2022-11-21 Cross-domain SFC deployment method of space-earth fusion network based on hybrid state synchronous DRL Active CN115834371B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211457628.5A CN115834371B (en) 2022-11-21 2022-11-21 Cross-domain SFC deployment method of space-earth fusion network based on hybrid state synchronous DRL

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211457628.5A CN115834371B (en) 2022-11-21 2022-11-21 Cross-domain SFC deployment method of space-earth fusion network based on hybrid state synchronous DRL

Publications (2)

Publication Number Publication Date
CN115834371A true CN115834371A (en) 2023-03-21
CN115834371B CN115834371B (en) 2024-05-03

Family

ID=85529757

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211457628.5A Active CN115834371B (en) 2022-11-21 2022-11-21 Cross-domain SFC deployment method of space-earth fusion network based on hybrid state synchronous DRL

Country Status (1)

Country Link
CN (1) CN115834371B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180026911A1 (en) * 2016-07-25 2018-01-25 Cisco Technology, Inc. System and method for providing a resource usage advertising framework for sfc-based workloads
CN110505099A (en) * 2019-08-28 2019-11-26 重庆邮电大学 A kind of service function chain dispositions method based on migration A-C study
CN113573320A (en) * 2021-07-06 2021-10-29 西安理工大学 SFC deployment method based on improved actor-critic algorithm in edge network
CN113660668A (en) * 2021-05-15 2021-11-16 西安电子科技大学 Seamless credible cross-domain routing system of heterogeneous converged network and control method thereof
CN114172820A (en) * 2021-11-26 2022-03-11 广东技术师范大学 Cross-domain SFC dynamic deployment method, device, computer equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180026911A1 (en) * 2016-07-25 2018-01-25 Cisco Technology, Inc. System and method for providing a resource usage advertising framework for sfc-based workloads
CN110505099A (en) * 2019-08-28 2019-11-26 重庆邮电大学 A kind of service function chain dispositions method based on migration A-C study
CN113660668A (en) * 2021-05-15 2021-11-16 西安电子科技大学 Seamless credible cross-domain routing system of heterogeneous converged network and control method thereof
CN113573320A (en) * 2021-07-06 2021-10-29 西安理工大学 SFC deployment method based on improved actor-critic algorithm in edge network
CN114172820A (en) * 2021-11-26 2022-03-11 广东技术师范大学 Cross-domain SFC dynamic deployment method, device, computer equipment and storage medium

Also Published As

Publication number Publication date
CN115834371B (en) 2024-05-03

Similar Documents

Publication Publication Date Title
CN110012516B (en) Low-orbit satellite routing strategy method based on deep reinforcement learning architecture
Zhao et al. Intelligent digital twin-based software-defined vehicular networks
Mousavinejad et al. Distributed cyber attacks detection and recovery mechanism for vehicle platooning
Wang et al. Synchronization of multi-layer networks: From node-to-node synchronization to complete synchronization
Yu et al. Distributed adaptive control for synchronization in directed complex networks
WO2021036414A1 (en) Co-channel interference prediction method for satellite-to-ground downlink under low earth orbit satellite constellation
Zhang et al. Sampled-data consensus of linear time-varying multiagent networks with time-varying topologies
Li et al. Resilient cooperative control for networked Lagrangian systems against DoS attacks
CN114124823B (en) Self-adaptive routing method, system and equipment oriented to high dynamic network topology
Xu et al. Living with artificial intelligence: A paradigm shift toward future network traffic control
Zhao et al. Decentralized Dynamic Event-Triggered $\mathcal {H} _ {\infty} $ Control for Nonlinear Systems With Unreliable Communication Channel and Limited Bandwidth
WO2022028926A1 (en) Offline simulation-to-reality transfer for reinforcement learning
CN115510316A (en) Privacy protection cross-domain recommendation system based on federal representation learning
CN113660676A (en) Base station flow prediction method, system, storage medium and equipment
CN115714741A (en) Routing decision method and system based on collaborative multi-agent reinforcement learning
CN115834371A (en) Space-ground converged network cross-domain SFC deployment method based on hybrid state synchronous DRL
Hu et al. Learning model parameters for decentralized schedule-driven traffic control
CN107749801A (en) A kind of virtual network function laying method based on population Incremental Learning Algorithm
Gao et al. Data-driven cooperative output regulation of multi-agent systems under distributed denial of service attacks
WO2018176768A1 (en) Network architecture of humanoid network and implementation method
Liu et al. General decentralized federated learning for communication-computation tradeoff
Chiu et al. Arc-based Traffic Assignment: Equilibrium Characterization and Learning
Huang et al. Bipartite multi-tracking in MASs with intermittent communication
Zuo et al. Distributed asynchronous consensus control of nonlinear multi-agent systems under directed switching topologies
CN113645055B (en) Implementation method suitable for multi-factor routing protocol in complex battlefield environment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant