CN111010341B - Overlay network routing decision method based on deep learning - Google Patents

Overlay network routing decision method based on deep learning Download PDF

Info

Publication number
CN111010341B
CN111010341B CN201911320959.2A CN201911320959A CN111010341B CN 111010341 B CN111010341 B CN 111010341B CN 201911320959 A CN201911320959 A CN 201911320959A CN 111010341 B CN111010341 B CN 111010341B
Authority
CN
China
Prior art keywords
network
data
overlay
matrix
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911320959.2A
Other languages
Chinese (zh)
Other versions
CN111010341A (en
Inventor
张旭
许刘泽
赵阳超
杨凯
马展
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University
Original Assignee
Nanjing University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University filed Critical Nanjing University
Priority to CN201911320959.2A priority Critical patent/CN111010341B/en
Publication of CN111010341A publication Critical patent/CN111010341A/en
Application granted granted Critical
Publication of CN111010341B publication Critical patent/CN111010341B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H04L45/121—Shortest path evaluation by minimising delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/02—Topology update or discovery
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/02—Topology update or discovery
    • H04L45/08—Learning-based routing, e.g. using neural networks or artificial intelligence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/14—Routing performance; Theoretical aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/131—Protocols for games, networked simulations or virtual reality

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses an overlay network routing decision method based on deep learning. The method comprises the following specific steps: s1, deploying overlay network routing nodes, and collecting bandwidth delay variation data sets; s2, making a data set for training, including a prediction network data set and a classification network data set selected by the overlay routing node, wherein the label of the prediction network is time delay and bandwidth data of the next moment, and the label calculation mode of the overlay routing node selecting the classification network is calculated on the overlay network topology formed by the overlay routing node by adopting a shortest path algorithm; s3, constructing a bandwidth delay prediction network based on a recurrent neural network (LSTM) and a coverage routing node selection classification network for selecting a coverage routing node; s4, training and optimizing a deep learning model; and S5, making a decision through the model. The method can greatly improve the decision response speed while ensuring the routing decision effect, especially when the coverage network node has huge scale.

Description

Overlay network routing decision method based on deep learning
Technical Field
The invention relates to the field of computer networks and video transmission, in particular to a real-time interactive video stream transmission scene with higher transmission delay requirement, and specifically relates to an overlay network routing decision method based on deep learning.
Background
With the development of the internet, network entertainment has widely penetrated into the daily life of people. But also makes the user's quality of experience highly correlated to network latency with respect to interactive attributes with network social entertainment. In a cloud gaming application scenario, if the network delay is greater than 50ms, 70% of the game players will perceive a significant delay. Furthermore, if network congestion occurs in the underlying routing path, the underlying routing path from the hosted game server to the user will experience a longer network delay. If the underlying routing connection fails, the cloud gaming service will experience a long recovery time, which will greatly affect the quality of experience.
Meanwhile, for the traditional route decision method (bottom layer route selection), in the process of route path selection, firstly, for any node, only bandwidth delay and the like of the current node can be observed, and parameters of the current node at the next moment cannot be well predicted, so that the optimal path of the current node cannot be selected. Secondly, for the decision method of using the overlay routing, because a lot of overlay routing nodes are designed and built, if the traditional shortest optimal path algorithm is used for calculation in the process of selecting the nodes, the calculation amount is too large at this time, and the real-time selection requirement of the routing cannot be met.
Disclosure of Invention
In view of the above situation, the present invention provides a method for overlay network routing decision based on deep learning, which can greatly improve the response speed of decision while ensuring the routing decision effect, especially when the overlay network node is large in scale.
The technical scheme adopted by the invention is as follows:
a method for making a route decision of an overlay network based on deep learning comprises the following steps:
s1, deploying overlay network routing nodes, and collecting a bandwidth matrix and a time delay matrix between every two overlay network routing nodes;
s2, making a data set (data, label) for training the model, wherein the data set comprises a prediction network data set and a classification network data set covering the routing node selection, the data is input data of the training model, and the label is label data corresponding to the input data;
s3, constructing a deep learning model, wherein the model consists of two deep neural networks connected in series: a bandwidth delay prediction network based on a recurrent neural network (LSTM) and a classification network selected by a coverage routing node based on a full-connection network;
s4, training and optimizing the deep learning model:
when the bandwidth delay prediction network is trained, the prediction network data set obtained in the step S2 is used as training data, the bandwidth matrix and the delay matrix at the current moment are used as input data, and the bandwidth matrix and the delay matrix at the next moment are used as label data, so as to train and optimize the bandwidth delay prediction network, so that the difference between the bandwidth delay matrix and the label data at the next moment obtained by prediction of the prediction network is as small as possible;
when training the classification network selected by the overlay routing nodes, the classification network data set selected by the overlay routing nodes obtained in step S2 is used as training data, the bandwidth matrix and the delay matrix at the current time are used as input data, and the shortest path algorithm is used to calculate the next-time optimal path on the overlay network topology formed by the overlay routing nodes
Figure BDA0002327135890000021
As label data, training a classification network selected by the optimized overlay routing node to ensure that the difference between the optimal path obtained by the network and the label data is as small as possible;
and cascading the two trained bandwidth delay prediction networks and the classification network selected by the overlay routing node to form a whole end-to-end deep learning model, and then training: using the data set obtained in step S2 as training data, using the bandwidth matrix and the delay matrix at the current time as input data, and using the optimal path at the next time
Figure BDA0002327135890000022
As label data, the difference between the optimal path obtained by the model and the label data is made as small as possible;
and S5, taking the bandwidth matrix and the delay matrix of the actual overlay network route as the input delay matrix and the bandwidth matrix of the deep learning model after training and optimizing in the step S4, and obtaining the optimal overlay network route path.
Compared with the traditional decision method, the method has the following beneficial effects:
(1) the invention selects a path with the shortest time delay from the user to the server under the condition of meeting the bandwidth requirement by utilizing the pre-arranged overlay network nodes, and can improve the speed and the quality of the user accessing the server.
(2) The invention selects the optimal overlay network routing path by utilizing an end-to-end neural network, and can greatly improve the decision response speed while ensuring the routing decision effect.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a diagram of the network input processing of the method of the present invention;
FIG. 3 is a block diagram of a network architecture for the method of the present invention;
FIG. 4 is a schematic diagram of the method of the present invention used by a user.
Detailed Description
The invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
In this embodiment, the method of the present invention is applied to a cloud game application scenario.
Referring to fig. 1, a method for making a route decision of an overlay network based on deep learning includes the following steps:
s1, deploying 10 overlay network routing nodes between the user and the cloud game server, wherein the overlay network routing nodes are x in sequence1,x2,…,x10At the present time tiConstantly collecting a bandwidth matrix B between every two coverage routing nodes:
Figure BDA0002327135890000023
any of them
Figure BDA0002327135890000031
Representing overlay network routing node xiRouting node x to overlay networkjThe bandwidth in between;
covering a time delay matrix D between every two routing nodes:
Figure BDA0002327135890000032
any of them
Figure BDA0002327135890000033
Representing overlay network routing node xiRouting node x to overlay networkjThe time delay between;
at this point, raw data for training the model may be obtained, including at time series t1,t2,…,tnThen, obtain the corresponding bandwidth matrix B1,B2,…,BnAnd a delay matrix D1,D2,…,Dn。
S2, data (data, label) for training the model is created, wherein the data is input data of the training model and the label is label data corresponding to the input data. The production data is divided into production prediction network training data and production overlay network node selection (classification network) training data. Where predicted network training data uses tiTime BiMatrix sum DiUsing the matrix as input data, using ti+1Time Bi+1Matrix sum Di+1The matrix is used as label data; predicting training data usage t of a networkiTime BiMatrix sum DiThe matrix is used as input data, and the optimal path obtained by calculation by using the shortest path algorithm
Figure BDA0002327135890000034
As tag data: :
Figure BDA0002327135890000035
(where n is a positive integer, the number of the n is the same as that of the overlay network routing nodes, and when y isiWhen the time is 0, x in the overlay network path with the shortest delay time under the condition of meeting the bandwidth requirement, which is obtained between the user and the server at the momentiNodes need not be used when yiWhen 1, x in the overlay network path with the shortest delay time under the condition of meeting the bandwidth requirement, which is obtained between the user and the server at the momentiThe node needs to be used.
S3, constructing a deep learning model:
and S31, constructing a bandwidth delay prediction network based on the LSTM. Wherein the number of LSTM neurons is 256.
And S32, constructing a classification network selected by the coverage routing node based on the full-connection network. Wherein the total connection network has three layers, each layer of network comprises input and output: the first layer is input n x 2 and output 256; the second level is input 256, output 256; the third level is input 256 and output n.
And S4, training and optimizing the deep learning model.
S41, training the prediction network of the step S31: using the predicted network training data obtained in step S2, t is usediTime BiMatrix sum DiUsing the matrix as input data, using ti+1Time Bi+1Matrix sum Di+1The matrix is used as label data, and the label data is expanded into vectors according to the processing method of FIG. 2 corresponding to FIG. 2
Figure BDA0002327135890000036
Figure BDA0002327135890000041
Using LpredictionLoss function (
Figure BDA0002327135890000042
Obtaining an output vector for the prediction network, k being the vector
Figure BDA0002327135890000043
Length of) training optimizes the LSTM-based bandwidth-delay prediction network such that the loss function LpredictionAs small as possible.
S42, training the classification network of step S32, using the classification network training data obtained in step S2, using tiTime BiMatrix sum DiThe matrix is used as input data, and the optimal path obtained by calculation by using the shortest path algorithm
Figure BDA0002327135890000044
As the data of the tag, it is,
Figure BDA0002327135890000045
using LClassificationLoss function (
Figure BDA0002327135890000046
Obtaining an output vector for the prediction network, k being the vector
Figure BDA0002327135890000047
Length of) training optimizes the classification network based on the overlay routing node selection of the fully-connected network such that the loss function LClassificationAs small as possible.
And S43, combining the networks trained in the steps S41 and S42 and then carrying out combined training. Using the training data obtained in step S2, t is usediTime BiMatrix sum DiUsing the matrix as input data, using ti+1Time Bi+1Matrix, Di+1The matrix is expanded into vectors
Figure BDA0002327135890000048
Optimal path calculated by shortest path algorithm
Figure BDA0002327135890000049
As tag data.
Lunion＝λ1LPrediction+λ2LClassification
Figure BDA00023271358900000410
Using LunionLoss function (
Figure BDA00023271358900000411
Obtaining an output vector for the prediction network, m being a vector
Figure BDA00023271358900000412
Length of (d);
Figure BDA00023271358900000413
obtaining an output vector for the prediction network, n being a vector
Figure BDA00023271358900000414
Length of) training the optimized joint model such that the joint network model LunionAs small as possible.
S5, deploying the model. And deploying the trained and optimized deep learning model into a DNS (domain name server), and firstly, requesting the DNS server to obtain connection with the cloud game server by a user in the cloud game experience process. Then, the DNS server uses the combined model obtained by training in step S4 as a decision method for the overlay network route to obtain B for recording the overlay network route at the current time and the previous timeiMatrix sum DiAnd the matrix is used for calculating to obtain an optimal network routing path according to the combined model, connecting the user and the cloud game server according to the obtained optimal network routing path and transmitting the cloud game data by using the path.

Claims (1)

1. A method for making a route decision of an overlay network based on deep learning is characterized by comprising the following steps:
s1, deploying overlay network routing nodes, and collecting a bandwidth matrix and a time delay matrix between every two overlay network routing nodes;
s2, making a data set (data, label) for training the model, wherein the data set comprises a prediction network data set and a classification network data set covering the routing node selection, the data is input data of the training model, and the label is label data corresponding to the input data;
s3, constructing a deep learning model, wherein the model consists of two deep neural networks connected in series: a bandwidth delay prediction network based on a recurrent neural network (LSTM) and a classification network selected by a coverage routing node based on a full-connection network;
s4, training and optimizing the deep learning model:
when the bandwidth delay prediction network is trained, the prediction network data set obtained in the step S2 is used as training data, the bandwidth matrix and the delay matrix at the current moment are used as input data, and the bandwidth matrix and the delay matrix at the next moment are used as label data, so as to train and optimize the bandwidth delay prediction network, so that the difference between the bandwidth delay matrix and the label data at the next moment obtained by prediction of the prediction network is as small as possible;
when training the classification network selected by the overlay routing nodes, the classification network data set selected by the overlay routing nodes obtained in step S2 is used as training data, the bandwidth matrix and the delay matrix at the current time are used as input data, and the shortest path algorithm is used to calculate the next-time optimal path on the overlay network topology formed by the overlay routing nodes
Figure FDA0002327135880000012
As label data, training a classification network selected by the optimized overlay routing node to ensure that the difference between the optimal path obtained by the network and the label data is as small as possible;
and cascading the two trained bandwidth delay prediction networks and the classification network selected by the overlay routing node to form a whole end-to-end deep learning model, and then training: using the data set obtained in step S2 as training data, using the bandwidth matrix and the delay matrix at the current time as input data, and using the optimal path at the next time
Figure FDA0002327135880000011
As label data, the difference between the optimal path obtained by the model and the label data is made as small as possible;
and S5, taking the bandwidth matrix and the delay matrix of the actual overlay network route as the input delay matrix and the bandwidth matrix of the deep learning model after training and optimizing in the step S4, and obtaining the optimal overlay network route path.
CN201911320959.2A 2019-12-19 2019-12-19 Overlay network routing decision method based on deep learning Active CN111010341B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911320959.2A CN111010341B (en) 2019-12-19 2019-12-19 Overlay network routing decision method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911320959.2A CN111010341B (en) 2019-12-19 2019-12-19 Overlay network routing decision method based on deep learning

Publications (2)

Publication Number Publication Date
CN111010341A CN111010341A (en) 2020-04-14
CN111010341B true CN111010341B (en) 2020-10-27

Family

ID=70116927

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911320959.2A Active CN111010341B (en) 2019-12-19 2019-12-19 Overlay network routing decision method based on deep learning

Country Status (1)

Country Link
CN (1) CN111010341B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111526556A (en) * 2020-04-29 2020-08-11 黄东 DTN (delay tolerant network) routing optimization method based on neural network
CN112702267B (en) * 2021-01-21 2023-04-07 广东工业大学 Distributed training routing method, system, storage medium and computer equipment
CN113177636A (en) * 2021-05-08 2021-07-27 中国电子科技集团公司第二十九研究所 Network dynamic routing method and system based on multiple constraint conditions
CN114448821A (en) * 2021-12-03 2022-05-06 航天科工网络信息发展有限公司 Intelligent routing method, device and network equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106953802A (en) * 2017-03-01 2017-07-14 浙江工商大学 A kind of network optimal route selection method based on deep learning
CN106953747A (en) * 2017-03-01 2017-07-14 浙江工商大学 A kind of SDN self-healing method based on deep learning
EP3264722A1 (en) * 2014-12-26 2018-01-03 System73, Inc. Method and system for adaptive virtual broadcasting of digital content
CN107911299A (en) * 2017-10-24 2018-04-13 浙江工商大学 A kind of route planning method based on depth Q study

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3264722A1 (en) * 2014-12-26 2018-01-03 System73, Inc. Method and system for adaptive virtual broadcasting of digital content
CN106953802A (en) * 2017-03-01 2017-07-14 浙江工商大学 A kind of network optimal route selection method based on deep learning
CN106953747A (en) * 2017-03-01 2017-07-14 浙江工商大学 A kind of SDN self-healing method based on deep learning
CN107911299A (en) * 2017-10-24 2018-04-13 浙江工商大学 A kind of route planning method based on depth Q study

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于深度学习的推荐系统研究综述;黄立威，江碧涛，吕守业，刘艳博，李德毅;《计算机学报》;20180731;第41卷(第7期);1619-1644 *

Also Published As

Publication number Publication date
CN111010341A (en) 2020-04-14

Similar Documents

Publication Publication Date Title
CN111010341B (en) Overlay network routing decision method based on deep learning
CN112491714B (en) Intelligent QoS route optimization method and system based on deep reinforcement learning in SDN environment
CN108540384B (en) Intelligent rerouting method and device based on congestion awareness in software defined network
CN111625361B (en) Joint learning framework based on cooperation of cloud server and IoT (Internet of things) equipment
CN112202672B (en) Network route forwarding method and system based on service quality requirement
CN102025620B (en) Cognitive network QoS (quality of service) guarantee method on basis of service differentiation
CN106716976A (en) Media session between network endpoints
US20190132422A1 (en) Method and system for server assignment using predicted network metrics
CN111245718A (en) Routing optimization method based on SDN context awareness
CN113315978B (en) Collaborative online video edge caching method based on federal learning
CN111211984B (en) Method and device for optimizing CDN network and electronic equipment
CN114650227A (en) Network topology construction method and system under layered federated learning scene
da Silva et al. Towards federated learning in edge computing for real-time traffic estimation in smart cities
CN113783944A (en) Video data processing method, device, system and equipment based on cloud edge cooperation
CN113783716A (en) Flow prediction method and device based on cloud edge collaborative framework
CN116527565A (en) Internet route optimization method and device based on graph convolution neural network
Li et al. Metaabr: A meta-learning approach on adaptative bitrate selection for video streaming
Zheng et al. A distributed learning architecture for semantic communication in autonomous driving networks for task offloading
CN103581329A (en) Construction method for topological structure based on clustered peer-to-peer network streaming media direct broadcast system
TWI792784B (en) Method and system for federated reinforcement learning based offloading optimization in edge computing
Phan et al. Utilitarian placement of composite services
Huo et al. Optimizing QoE of multiple users over DASH: A meta-learning approach
Mikhailenko et al. Analysis of the adaptive neural network router
CN118042018B (en) STSGCN-based intelligent multi-path data scheduling method
CN116596025B (en) Information popularity prediction method and system based on cascade relation diversity

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant