WO2023082554A1 - Adaptive network switching method and system, and storage medium - Google Patents

Adaptive network switching method and system, and storage medium Download PDF

Info

Publication number
WO2023082554A1
WO2023082554A1 PCT/CN2022/089236 CN2022089236W WO2023082554A1 WO 2023082554 A1 WO2023082554 A1 WO 2023082554A1 CN 2022089236 W CN2022089236 W CN 2022089236W WO 2023082554 A1 WO2023082554 A1 WO 2023082554A1
Authority
WO
WIPO (PCT)
Prior art keywords
network
access point
switching
deep
bandwidth
Prior art date
Application number
PCT/CN2022/089236
Other languages
French (fr)
Chinese (zh)
Inventor
路永玲
王真
胡成博
朱雪琼
杨景刚
贾骏
孙蓉
刘子全
薛海
张东磊
Original Assignee
国网江苏省电力有限公司电力科学研究院
国网江苏省电力有限公司
江苏省电力试验研究院有限公司
国网智能电网研究院有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202111325359.2A external-priority patent/CN114125962B/en
Application filed by 国网江苏省电力有限公司电力科学研究院, 国网江苏省电力有限公司, 江苏省电力试验研究院有限公司, 国网智能电网研究院有限公司 filed Critical 国网江苏省电力有限公司电力科学研究院
Priority to KR1020227030640A priority Critical patent/KR20230070405A/en
Publication of WO2023082554A1 publication Critical patent/WO2023082554A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W36/00—Hand-off or reselection arrangements
    • H04W36/14—Reselecting a network or an air interface
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W36/00—Hand-off or reselection arrangements
    • H04W36/0005—Control or signalling for completing the hand-off
    • H04W36/0055—Transmission or use of information for re-establishing the radio link
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W36/00—Hand-off or reselection arrangements
    • H04W36/24—Reselection being triggered by specific parameters
    • H04W36/30—Reselection being triggered by specific parameters by measured or perceived connection quality data

Definitions

  • the invention relates to an adaptive network switching method, system and storage medium, and belongs to the technical field of wireless networks.
  • heterogeneous wireless networks with multi-network integration and seamless roaming will become an inevitable trend.
  • network handover technology is one of the key technologies to ensure session continuity when devices move across heterogeneous networks, and has important research significance.
  • RSS Received Signal Strength
  • MCDM Multi-criteria Decision Making
  • the invention provides an adaptive network switching method, system and storage medium, which solves the problems of frequent equipment switching and unreasonable switching criteria caused by the traditional method.
  • a method for adaptive network switching comprising:
  • the network environment state parameters are used as input, and the pre-trained first deep Q network and second deep Q network are used to obtain the selected network and the access point of the selected network, according to the selection Connected network and select the access point connected to the network to switch the network;
  • the network standard of the device is a single network standard and the business type of the device is non-fixed business
  • the network environment status parameters are used as input, and the pre-trained second depth Q network is used to obtain the network access point for the selected connection. Access point for network switching.
  • the status information of the device includes the change value of the received power of the device, the change value of the received delay of the device and the inherent parameters of the device.
  • the device network standard supports multiple network standards, use the pre-trained first deep Q network and second deep Q network to obtain the selected network and the access point of the selected network, according to the selected network and the selected connection network Access point for network switching, including:
  • the network standard of the device supports multiple network standards, use the pre-trained first depth Q network to obtain the network to be selected for connection, and use the pre-trained second depth Q network to obtain the access point to select the network to connect to. and select the access point connected to the network to switch the network.
  • the input of the first deep Q network is the network environment state parameters, including:
  • the device's demand matrix for network bandwidth, delay, bit error rate, and jitter is a parameter that specifies the delay.
  • the network switching reward in the first deep Q network is:
  • r 1 is the network switching reward in the first deep Q network, Select weights for networks with bandwidth when switching between networks, Select weights for networks with delays in network switching, Select the weight for the network for the bit error rate when the network is switched,
  • the network selection weight for the jitter during network switching f 1B (S 1 ,n) is the network selection revenue function of bandwidth during network switching, f 1 ⁇ (S 1 ,n) is the network selection revenue function of network switching delay, f 1e (S 1 ,n) is the network selection benefit function of bit error rate during network switching, f 1J (S 1 ,n) is the network selection benefit function of jitter during network switching, and S 1 is the network environment input to the first deep Q network State parameter set.
  • B n is the bandwidth provided by wireless network n
  • ⁇ n is the delay provided by wireless network n
  • e n is the bit error rate provided by wireless network n
  • J n is the jitter provided by wireless network n
  • qB is the bandwidth required by wireless network n
  • q ⁇ is the delay required by wireless network n
  • qe is the bit error rate required by wireless network n
  • qJ is the jitter required by wireless network n.
  • the input of the second deep Q network is the network environment state parameters, including:
  • the device receives the receiving power of each access point
  • the device's demand matrix for network bandwidth, delay, bit error rate, and jitter is a parameter that specifies the delay.
  • the network switching reward in the second deep Q network is:
  • r 2 is the network switching reward in the second deep Q network, Select weights for networks with bandwidth when selecting access points, Select weights for networks with delays when selecting access points, Select the weight for the network for the bit error rate when selecting an access point, Select the weight for the jitter network when selecting the access point, f 2B (S 2 ,m) is the network selection revenue function of the bandwidth when selecting the access point, and f 2 ⁇ (S 2 ,m) is the network delay when selecting the access point Select the income function, f 2e (S 2 ,m) is the network selection income function of bit error rate when selecting the access point, f 2J (S 2 ,m) is the network selection income function of jitter when selecting the access point, S 2 is the network environment state parameter set input to the second deep Q network, P m is the received power of access point m, and P th is the sensitivity of the received power
  • B′ m is the bandwidth provided by the access point m
  • ⁇ ′ m is the delay provided by the access point m
  • e′ m is the bit error rate provided by the access point m
  • J′ m is the access point
  • qB' is the network bandwidth required by access point m
  • q ⁇ ' is the network delay required by access point m
  • qe' is the network bit error rate required by access point m
  • qJ' is the network bandwidth required by access point m.
  • An adaptive network switching system comprising:
  • RBF neural network module Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
  • the first switching module if the network standard of the device supports multiple network standards, the pre-trained first deep Q network and the second deep Q network are used to obtain the selected network and the access point of the selected network, according to the selected network And select the access point connected to the network to switch the network;
  • the second switching module if the device network standard is a single network standard and the device business type is non-fixed business, use the pre-trained second depth Q network to obtain the selected network access point, and select the connected network access point to perform network switching.
  • the first switching module if the network standard of the device supports multi-network standard, the pre-trained first deep Q network is used to obtain the network to be selected for connection, and the pre-trained second deep Q network is used to obtain the access point of the selected network, Network switching is performed according to the network selected for connection and the access point selected for connection to the network.
  • a computer-readable storage medium storing one or more programs including instructions that, when executed by a computing device, cause the computing device to perform an adaptive network switching method.
  • the present invention judges the service type of the equipment and the network standard of the equipment based on the RBF neural network, thereby determining the depth Q network used for switching, and using the current heterogeneous network or the environment state of the network access point as the depth Q network Input, obtain the optimal network and access point, avoid the wrong handover caused by the single handover index of the traditional handover algorithm, reduce the number of equipment handovers, and handover is more reasonable.
  • Fig. 1 is the flowchart of the inventive method
  • Figure 2 is a preset rule table
  • Figure 4 shows a heterogeneous wireless network scenario.
  • a method for adaptive network handover includes the following steps:
  • Step 1 Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment service type and equipment network standard;
  • Step 2 if the network standard of the device supports multiple network standards, the network environment state parameters are used as input, and the pre-trained first depth Q network and second depth Q network are used to obtain the network to be connected and the access point to be connected to the network , switch the network according to the selected network and the selected access point to connect to the network;
  • Step 3 If the device network standard is a single network standard and the device service type is non-fixed service, the network environment status parameters are used as input, and the pre-trained second depth Q network is used to obtain the network access point for the selected connection. According to the selected The connected network access point performs network switching.
  • the above method is based on the RBF neural network to judge the device service type and device network standard, thereby determining the deep Q network used for switching, and using the current heterogeneous network or the environment state of the network access point as the input of the deep Q network to obtain the optimal network and access point.
  • the entry point avoids false switching caused by the single switching index of the traditional switching algorithm, reduces the number of device switching times, and makes the switching more reasonable.
  • Training RBF neural network Initialize the RBF neural network with 3 input nodes (received power change value, delay change value and device inherent parameters), number of hidden nodes (the number of hidden nodes is determined by the error back propagation algorithm) and the number of output nodes 2 (output device service type and device network standard); the actual measured power in the wireless network, delay variation, device intrinsic parameters, device service type and device network standard are used as samples to train the RBF neural network.
  • the RBF neural network When the RBF neural network is in use, directly input the status information of the device, that is, the change value of the device receiving power, the change value of the device receiving delay, and the inherent parameters of the device, into the RBF neural network to obtain the device service type and device network standard, using the formula can be expressed as:
  • RBF in ⁇ P, ⁇ ,F ⁇
  • RBF in and RBF out are the input and output of the RBF neural network, respectively
  • ⁇ P, ⁇ , and F are the device receiving power change value, device receiving delay change value and device inherent parameters
  • State1 is the device service type
  • State2 indicates the network standard of the device
  • the device can switch between multiple networks. If the device supports a single network standard, the device can only switch in a single network, that is, access Click Switch.
  • the preset rules in Figure 2 can be used to determine the network used for network switching.
  • the details can be as follows:
  • the device network standard supports multiple network standards, no matter whether the device business type is fixed business (such as temperature, humidity, pressure sensor, etc. transmission business) or mobile business (such as smart label, operator smart helmet, etc. transmission business), First use the first depth Q network to obtain the network selected for connection, that is, the network after switching, and then use the second depth Q network to obtain the access point of the selected connection network, that is, the access point of the network after switching;
  • the device business type such as temperature, humidity, pressure sensor, etc. transmission business
  • mobile business such as smart label, operator smart helmet, etc. transmission business
  • the device network standard is a single network standard and the device service type is a fixed service, then the device cannot perform network switching; if the device network standard is a single network standard and the device service type is a non-fixed service (ie mobile service) , directly adopting the second deep Q network to obtain the network access point selected for connection, that is, the switched network access point.
  • a non-fixed service ie mobile service
  • the above-mentioned first deep Q-network is only responsible for vertical switching (i.e. inter-network switching), and the second deep Q-network is only responsible for access point switching.
  • the first deep Q-network and the second deep-Q network use the same deep Q-network, including real
  • the Q-network and the target Q-network also have network environment state parameters as their inputs, but because the purposes of the two are different, the input parameters and network switching reward functions are different.
  • the actual Q network is interacted with the wireless network environment, that is, the network environment state parameters are input into the actual Q network, wherein, the network environment state parameters input into the first deep Q network can be:
  • S 1 is the network environment state parameter set input to the first depth Q network
  • B n is the bandwidth provided by wireless network n
  • ⁇ n is the time delay provided by wireless network n
  • e n is the time delay provided by wireless network n
  • Bit error rate J n is the jitter provided by wireless network n
  • X is the demand matrix of equipment for network bandwidth, delay, bit error rate and jitter.
  • the network environment state parameter input into the second depth Q network can be:
  • the device receives the receiving power of each access point
  • s 2 is the network environment state parameter set input into the second depth Q network
  • P m is the received power of access point m
  • B′ m is the bandwidth provided by access point m
  • ⁇ ′ m is the access point m
  • the provided delay e' m is the bit error rate provided by the access point m
  • J' m is the jitter provided by the access point m.
  • a 1 and a 2 are network switching actions and network access point switching actions respectively
  • ⁇ 1 and ⁇ 1 are the parameters of the first deep Q network and the second deep Q network respectively
  • ⁇ is a generated random number
  • ⁇ is the probability of exploration.
  • the environment By executing the action, the environment will return to the state S' 1 or S' 2 at the next moment, and the reward for the device performing the network switching action.
  • the network switching reward function can be expressed by the following formula:
  • r 1 is the network switching reward in the first deep Q network, Select weights for networks with bandwidth when switching between networks, Select weights for networks with delays in network switching, Select the weight for the network for the bit error rate when the network is switched,
  • the network selection weight for the jitter during network switching f 1B (S 1 ,n) is the network selection revenue function of bandwidth during network switching, f 1 ⁇ (S 1 ,n) is the network selection revenue function of network switching delay, f 1e (S 1 ,n) is the network selection revenue function of bit error rate during network switching, f 1J (S 1 ,n) is the network selection revenue function of jitter during network switching, qB is the bandwidth required by wireless network n, and q ⁇ is the wireless The delay required by network n, qe is the bit error rate required by wireless network n, and qJ is the jitter required by wireless network n.
  • the network switching reward function can be expressed by the following formula:
  • r 2 is the network switching reward in the second deep Q network, Select weights for networks with bandwidth when selecting access points, Select weights for networks with delays when selecting access points, Select weights for the network for bit error rates when selecting access points, It is the network selection weight of the jitter when selecting the access point, f 2B (S 2 ,m) is the network selection benefit function of the bandwidth when selecting the access point, and f 2 ⁇ (S 2 ,m) is the time delay when selecting the access point Network selection revenue function, f 2e (S 2 ,m) is the network selection revenue function of bit error rate when selecting an access point, f 2J (S 2 ,m) is the network selection revenue function of jitter when selecting an access point, P th is the sensitivity of receiving power, qB' is the network bandwidth required by access point m, q ⁇ ' is the network delay required by access point m, qe' is the network bit error rate required by access point m, and qJ' is the network bandwidth required by access point m.
  • the above four-dimensional data (S 1 , a 1 , r 1 , S′ 1 ) of the first deep Q network or the four-dimensional data (S 2 , a 2 , r 2 , S′ 2 ) of the second deep Q network will be stored in the empirical In the pool, the deep Q network is trained through the data in the experience pool.
  • a' indicates the switching action of the next state
  • ⁇ - indicates the network parameters of the target Q network
  • the subscript 1 indicates that it corresponds to the first deep Q network
  • 2 indicates that it corresponds to the second deep Q network.
  • the target Q value can be updated as:
  • ⁇ represents the discount factor and r represents the reward.
  • the loss function can be calculated using the updated target Q value, as shown in Figure 3, the formula can be expressed as:
  • ⁇ is the deep Q network parameter
  • E is the expected operation
  • the network parameters of the real Q network are updated through the backpropagation of the loss function. Specifically, the network parameters of the current real Q network are copied to the target Q network every certain number of steps.
  • the above method can be implemented in the heterogeneous wireless network scenario shown in Figure 4.
  • the scenario is a smart factory, substation, underground pipe corridor or large sports field, etc.
  • first construct and train the RBF neural network and deep Q network and then the trained The network is built into devices with computing power. Every time period, the RBF neural network will judge whether the device currently supports multi-network standards and the type of business transmitted according to the real-time device status information, and select the corresponding deep Q network to judge the switching action through the method in Figure 2.
  • the above method is oriented to different devices and different services in the wireless sensor network, combines the RBF neural network and the deep Q network, and judges the device service type and device network standard based on the RBF neural network, so as to determine the deep Q network used for switching.
  • a deep Q network is responsible for switching different network standards, and a second deep Q network is responsible for switching different access points under the same network.
  • Both deep Q-networks are trained through the network state parameters and constructed reward functions, and the current heterogeneous network or the environmental state of the network access point is used as the input of the deep Q-network to judge the network switching and obtain the optimal network and access points. point, avoiding the wrong switching caused by the single switching index of the traditional switching algorithm, reducing the number of equipment switching, and switching more reasonable.
  • the method can realize accurate and effective network switching of different networks, and improve the service quality of users.
  • an adaptive network switching system including:
  • RBF neural network module Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
  • the first switching module if the network standard of the device supports multi-network standard, the pre-trained first deep Q network is used to obtain the network to be selected for connection, and the pre-trained second deep Q network is used to obtain the access point of the selected network, Perform network switching according to the selected network and the access point selected to connect to the network;
  • the second switching module if the device network standard is a single network standard and the device business type is non-fixed business, use the pre-trained second depth Q network to obtain the selected network access point, and select the connected network access point to perform network switching.
  • the present invention also discloses a computer-readable storage medium that stores one or more programs, and the one or more programs include instructions that, when executed by a computing device, cause the computing The device implements an adaptive network switching method.
  • the present invention also discloses a computing device, including one or more processors, one or more memories, and one or more programs, wherein one or more programs are stored in the one or more stored in the memory and configured to be executed by the one or more processors, the one or more programs include instructions for executing the adaptive network handover method.
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

Disclosed in the present invention are a self-adaptive network switching method and system, and a storage medium. According to the present invention, a device service type and a device network standard are determined on the basis of a RBF neural network so as to determine a deep Q network used for switching, and the environmental state of the current heterogeneous network or network access point is taken as an input into the deep Q network to obtain an optimal network and an optimal access point. Therefore, false switching caused by the fact that a switching index of a traditional switching algorithm is undiversified is avoided, the number of instances of device switching is reduced, and switching is more rational.

Description

一种自适应网络切换方法、系统及存储介质An adaptive network switching method, system and storage medium 技术领域technical field
本发明涉及一种自适应网络切换方法、系统及存储介质，属于无线网络技术领域。The invention relates to an adaptive network switching method, system and storage medium, and belongs to the technical field of wireless networks.
背景技术Background technique
随着未来各种应用场景中多种无线网络的部署，多网融合发展和无缝漫游的异构无线网络将成为必然趋势。作为异构无线网络移动性管理技术的核心，网络切换技术是在设备跨异构网络移动时确保会话连续性的关键技术之一，具有重要的研究意义。With the deployment of multiple wireless networks in various application scenarios in the future, heterogeneous wireless networks with multi-network integration and seamless roaming will become an inevitable trend. As the core of heterogeneous wireless network mobility management technology, network handover technology is one of the key technologies to ensure session continuity when devices move across heterogeneous networks, and has important research significance.
目前传统的切换方法包括基于接受信号强度(Received Signal Strength，RSS)算法和基于多准则决策(Multi-criteria Decision Making，MCDM)算法；其中，RSS算法会导致设备的频繁切换，影响用户服务质量，MCDM算法由于多种决策标准相互依赖和交互影响切换判决准则的相对权重，最终导致切换不合理。At present, traditional handover methods include Received Signal Strength (RSS) algorithm and Multi-criteria Decision Making (MCDM) algorithm; among them, the RSS algorithm will cause frequent handover of equipment, affecting the quality of user service. In MCDM algorithm, various decision-making criteria depend on each other and interact to influence the relative weight of handover decision criteria, which eventually leads to unreasonable handover.
发明内容Contents of the invention
本发明提供了一种自适应网络切换方法、系统及存储介质，解决了传统方法导致设备切换频繁和切换判据不合理的问题。The invention provides an adaptive network switching method, system and storage medium, which solves the problems of frequent equipment switching and unreasonable switching criteria caused by the traditional method.
为了解决上述技术问题，本发明所采用的技术方案是：In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:
一种自适应网络切换方法，包括：A method for adaptive network switching, comprising:
将网络环境下设备的状态信息输入预先训练的RBF神经网络，获得设备业务类型和设备网络制式；Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
若设备网络制式为支持多网络制式，将网络环境状态参数作为输入，采用 预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换；If the device network standard supports multiple network standards, the network environment state parameters are used as input, and the pre-trained first deep Q network and second deep Q network are used to obtain the selected network and the access point of the selected network, according to the selection Connected network and select the access point connected to the network to switch the network;
若设备网络制式为单网络制式且设备业务类型为非固定类业务，将网络环境状态参数作为输入，采用预先训练的第二深度Q网络，获得选择连接的网络接入点，根据选择连接的网络接入点，进行网络切换。If the network standard of the device is a single network standard and the business type of the device is non-fixed business, the network environment status parameters are used as input, and the pre-trained second depth Q network is used to obtain the network access point for the selected connection. Access point for network switching.
设备的状态信息包括设备接收功率变化值、设备接收时延变化值和设备固有参数。The status information of the device includes the change value of the received power of the device, the change value of the received delay of the device and the inherent parameters of the device.
若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换，包括：If the device network standard supports multiple network standards, use the pre-trained first deep Q network and second deep Q network to obtain the selected network and the access point of the selected network, according to the selected network and the selected connection network Access point for network switching, including:
若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络，获得选择连接的网络，采用预先训练的第二深度Q网络，获得选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换。If the network standard of the device supports multiple network standards, use the pre-trained first depth Q network to obtain the network to be selected for connection, and use the pre-trained second depth Q network to obtain the access point to select the network to connect to. and select the access point connected to the network to switch the network.
第一深度Q网络的输入为网络环境状态参数，包括：The input of the first deep Q network is the network environment state parameters, including:
环境中各无线网络的带宽、时延、误码率和抖动；The bandwidth, delay, bit error rate and jitter of each wireless network in the environment;
设备对网络带宽、时延、误码率和抖动的需求矩阵。The device's demand matrix for network bandwidth, delay, bit error rate, and jitter.
第一深度Q网络中网络切换奖励为：The network switching reward in the first deep Q network is:
Figure PCTCN2022089236-appb-000001
Figure PCTCN2022089236-appb-000001
其中，r 1为第一深度Q网络中网络切换奖励，
Figure PCTCN2022089236-appb-000002
为网络切换时带宽的网络选择权重，
Figure PCTCN2022089236-appb-000003
为网络切换时时延的网络选择权重，
Figure PCTCN2022089236-appb-000004
为网络切换时误码率的网络选择权重，
Figure PCTCN2022089236-appb-000005
为网络切换时抖动的网络选择权重，f 1B(S 1,n)为网络切换时 带宽的网络选择收益函数，f 1τ(S 1,n)为网络切换时时延的网络选择收益函数，f 1e(S 1,n)为网络切换时误码率的网络选择收益函数，f 1J(S 1,n)为网络切换时抖动的网络选择收益函数，S 1为输入第一深度Q网络的网络环境状态参数集。 Among them, r 1 is the network switching reward in the first deep Q network,
Figure PCTCN2022089236-appb-000002
Select weights for networks with bandwidth when switching between networks,
Figure PCTCN2022089236-appb-000003
Select weights for networks with delays in network switching,
Figure PCTCN2022089236-appb-000004
Select the weight for the network for the bit error rate when the network is switched,
Figure PCTCN2022089236-appb-000005
The network selection weight for the jitter during network switching, f 1B (S 1 ,n) is the network selection revenue function of bandwidth during network switching, f 1τ (S 1 ,n) is the network selection revenue function of network switching delay, f 1e (S 1 ,n) is the network selection benefit function of bit error rate during network switching, f 1J (S 1 ,n) is the network selection benefit function of jitter during network switching, and S 1 is the network environment input to the first deep Q network State parameter set.
Figure PCTCN2022089236-appb-000006
Figure PCTCN2022089236-appb-000006
Figure PCTCN2022089236-appb-000007
Figure PCTCN2022089236-appb-000007
Figure PCTCN2022089236-appb-000008
Figure PCTCN2022089236-appb-000008
Figure PCTCN2022089236-appb-000009
Figure PCTCN2022089236-appb-000009
其中，B n为无线网络n所提供的带宽，τ n为无线网络n所提供的时延，e n为无线网络n所提供的误码率，J n为无线网络n所提供的抖动，qB为无线网络n需求的带宽，qτ为无线网络n需求的时延，qe为无线网络n需求的误码率，qJ为无线网络n需求的抖动。 Among them, B n is the bandwidth provided by wireless network n, τ n is the delay provided by wireless network n, e n is the bit error rate provided by wireless network n, J n is the jitter provided by wireless network n, qB is the bandwidth required by wireless network n, qτ is the delay required by wireless network n, qe is the bit error rate required by wireless network n, and qJ is the jitter required by wireless network n.
第二深度Q网络的输入为网络环境状态参数，包括：The input of the second deep Q network is the network environment state parameters, including:
网络中各接入点的带宽、时延、误码率和抖动；Bandwidth, delay, bit error rate and jitter of each access point in the network;
设备接收各接入点的接收功率；The device receives the receiving power of each access point;
设备对网络带宽、时延、误码率和抖动的需求矩阵。The device's demand matrix for network bandwidth, delay, bit error rate, and jitter.
第二深度Q网络中网络切换奖励为：The network switching reward in the second deep Q network is:
Figure PCTCN2022089236-appb-000010
Figure PCTCN2022089236-appb-000010
其中，r 2为第二深度Q网络中网络切换奖励，
Figure PCTCN2022089236-appb-000011
为选择接入点时带宽的网络选择权重，
Figure PCTCN2022089236-appb-000012
为选择接入点时时延的网络选择权重，
Figure PCTCN2022089236-appb-000013
为选择接入点时误 码率的网络选择权重，
Figure PCTCN2022089236-appb-000014
为选择接入点时抖动的网络选择权重，f 2B(S 2,m)为选择接入点时带宽的网络选择收益函数，f 2τ(S 2,m)为选择接入点时时延的网络选择收益函数，f 2e(S 2,m)为选择接入点时误码率的网络选择收益函数，f 2J(S 2,m)为选择接入点时抖动的网络选择收益函数，S 2为输入第二深度Q网络的网络环境状态参数集，P m为接入点m的接收功率，P th为接收功率的灵敏度 Among them, r 2 is the network switching reward in the second deep Q network,
Figure PCTCN2022089236-appb-000011
Select weights for networks with bandwidth when selecting access points,
Figure PCTCN2022089236-appb-000012
Select weights for networks with delays when selecting access points,
Figure PCTCN2022089236-appb-000013
Select the weight for the network for the bit error rate when selecting an access point,
Figure PCTCN2022089236-appb-000014
Select the weight for the jitter network when selecting the access point, f 2B (S 2 ,m) is the network selection revenue function of the bandwidth when selecting the access point, and f 2τ (S 2 ,m) is the network delay when selecting the access point Select the income function, f 2e (S 2 ,m) is the network selection income function of bit error rate when selecting the access point, f 2J (S 2 ,m) is the network selection income function of jitter when selecting the access point, S 2 is the network environment state parameter set input to the second deep Q network, P m is the received power of access point m, and P th is the sensitivity of the received power
Figure PCTCN2022089236-appb-000015
Figure PCTCN2022089236-appb-000015
Figure PCTCN2022089236-appb-000016
Figure PCTCN2022089236-appb-000016
Figure PCTCN2022089236-appb-000017
Figure PCTCN2022089236-appb-000017
Figure PCTCN2022089236-appb-000018
Figure PCTCN2022089236-appb-000018
其中，B′ m为接入点m所提供的带宽，τ′ m为接入点m所提供的时延，e′ m为接入点m所提供的误码率，J′ m为接入点m所提供的抖动，qB′为接入点m需求的网络带宽，qτ′为接入点m需求的网络时延，qe′为接入点m需求的网络误码率，qJ′为接入点m需求的网络抖动。 Among them, B′ m is the bandwidth provided by the access point m, τ′ m is the delay provided by the access point m, e′ m is the bit error rate provided by the access point m, and J′ m is the access point The jitter provided by point m, qB' is the network bandwidth required by access point m, qτ' is the network delay required by access point m, qe' is the network bit error rate required by access point m, and qJ' is the network bandwidth required by access point m. Network jitter required by entry point m.
一种自适应网络切换系统，包括：An adaptive network switching system, comprising:
RBF神经网络模块：将网络环境下设备的状态信息输入预先训练的RBF神经网络，获得设备业务类型和设备网络制式；RBF neural network module: Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
第一切换模块：若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换；The first switching module: if the network standard of the device supports multiple network standards, the pre-trained first deep Q network and the second deep Q network are used to obtain the selected network and the access point of the selected network, according to the selected network And select the access point connected to the network to switch the network;
第二切换模块：若设备网络制式为单网络制式且设备业务类型为非固定类 业务，采用预先训练的第二深度Q网络，获得选择连接的网络接入点，根据选择连接的网络接入点，进行网络切换。The second switching module: if the device network standard is a single network standard and the device business type is non-fixed business, use the pre-trained second depth Q network to obtain the selected network access point, and select the connected network access point to perform network switching.
第一切换模块：若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络，获得选择连接的网络，采用预先训练的第二深度Q网络，获得选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换。The first switching module: if the network standard of the device supports multi-network standard, the pre-trained first deep Q network is used to obtain the network to be selected for connection, and the pre-trained second deep Q network is used to obtain the access point of the selected network, Network switching is performed according to the network selected for connection and the access point selected for connection to the network.
一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序包括指令，所述指令当由计算设备执行时，使得所述计算设备执行自适应网络切换方法。A computer-readable storage medium storing one or more programs including instructions that, when executed by a computing device, cause the computing device to perform an adaptive network switching method.
本发明所达到的有益效果：本发明基于RBF神经网络判断设备业务类型和设备网络制式，从而确定切换所使用的深度Q网络，将当前异构网络或网络接入点的环境状态作为深度Q网络输入，获得最优网络及接入点，避免了因传统切换算法切换指标单一而引发的误切换，减少设备切换次数，切换更加合理。The beneficial effects achieved by the present invention: the present invention judges the service type of the equipment and the network standard of the equipment based on the RBF neural network, thereby determining the depth Q network used for switching, and using the current heterogeneous network or the environment state of the network access point as the depth Q network Input, obtain the optimal network and access point, avoid the wrong handover caused by the single handover index of the traditional handover algorithm, reduce the number of equipment handovers, and handover is more reasonable.
附图说明Description of drawings
图1为本发明方法的流程图；Fig. 1 is the flowchart of the inventive method;
图2为预设规则表；Figure 2 is a preset rule table;
图3损失函数构造示意图；Figure 3 Schematic diagram of loss function construction;
图4为异构无线网络场景。Figure 4 shows a heterogeneous wireless network scenario.
具体实施方式Detailed ways
下面结合附图对本发明作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solution of the present invention more clearly, but not to limit the protection scope of the present invention.
如图1所示，一种自适应网络切换方法，包括以下步骤：As shown in Figure 1, a method for adaptive network handover includes the following steps:
步骤1，将网络环境下设备的状态信息输入预先训练的RBF神经网络，获得设备业务类型和设备网络制式；Step 1. Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment service type and equipment network standard;
步骤2，若设备网络制式为支持多网络制式，将网络环境状态参数作为输入，采用预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换； Step 2, if the network standard of the device supports multiple network standards, the network environment state parameters are used as input, and the pre-trained first depth Q network and second depth Q network are used to obtain the network to be connected and the access point to be connected to the network , switch the network according to the selected network and the selected access point to connect to the network;
步骤3，若设备网络制式为单网络制式且设备业务类型为非固定类业务，将网络环境状态参数作为输入，采用预先训练的第二深度Q网络，获得选择连接的网络接入点，根据选择连接的网络接入点，进行网络切换。Step 3. If the device network standard is a single network standard and the device service type is non-fixed service, the network environment status parameters are used as input, and the pre-trained second depth Q network is used to obtain the network access point for the selected connection. According to the selected The connected network access point performs network switching.
上述方法基于RBF神经网络判断设备业务类型和设备网络制式，从而确定切换所使用的深度Q网络，将当前异构网络或网络接入点的环境状态作为深度Q网络输入，获得最优网络及接入点，避免了因传统切换算法切换指标单一而引发的误切换，减少设备切换次数，切换更加合理。The above method is based on the RBF neural network to judge the device service type and device network standard, thereby determining the deep Q network used for switching, and using the current heterogeneous network or the environment state of the network access point as the input of the deep Q network to obtain the optimal network and access point. The entry point avoids false switching caused by the single switching index of the traditional switching algorithm, reduces the number of device switching times, and makes the switching more reasonable.
在实施上述方法之前，需要预先训练RBF神经网络、第一深度Q网络和第二深度Q网络。Before implementing the above method, it is necessary to pre-train the RBF neural network, the first deep Q network and the second deep Q network.
训练RBF神经网络：初始化RBF神经网络输入节点3个(接收功率变化值、时延变化值以及设备固有参数)、隐藏节点数个(由误差反向传播算法决定隐藏节点数量)及输出节点数量2个(输出设备业务类型和设备网络制式)；将无线网络中的实际测量的功率、时延变化量、设备固有参数、设备业务类型和设备网络制式作为样本，训练RBF神经网络。Training RBF neural network: Initialize the RBF neural network with 3 input nodes (received power change value, delay change value and device inherent parameters), number of hidden nodes (the number of hidden nodes is determined by the error back propagation algorithm) and the number of output nodes 2 (output device service type and device network standard); the actual measured power in the wireless network, delay variation, device intrinsic parameters, device service type and device network standard are used as samples to train the RBF neural network.
RBF神经网络在使用时，直接将设备的状态信息，即设备接收功率变化值、设备接收时延变化值和设备固有参数，输入RBF神经网络，即可获得设备业务 类型和设备网络制式，用公式可以表达为：When the RBF neural network is in use, directly input the status information of the device, that is, the change value of the device receiving power, the change value of the device receiving delay, and the inherent parameters of the device, into the RBF neural network to obtain the device service type and device network standard, using the formula can be expressed as:
RBF in＝{ΔP,Δτ，F} RBF in ={ΔP,Δτ,F}
RBF out＝{State1，State2},State1＝0,1,State2＝0,1 RBF out = {State1, State2}, State1 = 0, 1, State2 = 0, 1
其中，RBF in、RBF out分别为RBF神经网络的输入和输出，ΔP、Δτ、F分别为设备接收功率变化值、设备接收时延变化值和设备固有参数，State1为设备业务类型，State1＝1表示设备业务类型为移动类业务，State1＝0表示设备业务类型为固定类业务，State2为设备网络制式，State2＝1表示设备网络制式为支持多网络制式，State1＝0表示设备网络制式为单网络制式。 Among them, RBF in and RBF out are the input and output of the RBF neural network, respectively, ΔP, Δτ, and F are the device receiving power change value, device receiving delay change value and device inherent parameters, State1 is the device service type, State1=1 Indicates that the service type of the device is a mobile service, State1=0 indicates that the service type of the device is a fixed service, State2 indicates the network standard of the device, State2=1 indicates that the network standard of the device supports multiple networks, and State1=0 indicates that the network standard of the device is a single network format.
例如环境中有5G、WiFi、LoRa，若设备支持多网络制式时，那么该设备可以实现多个网络的切换，若设备支持单网络制式，那么设备仅能在单个网络中切换，即进行接入点切换。For example, there are 5G, WiFi, and LoRa in the environment. If the device supports multiple network standards, the device can switch between multiple networks. If the device supports a single network standard, the device can only switch in a single network, that is, access Click Switch.
因此获得设备业务类型和设备网络制式后，可采用图2中预设的规则，确定网络切换所采用的网络，具体可以如下：Therefore, after obtaining the device service type and device network standard, the preset rules in Figure 2 can be used to determine the network used for network switching. The details can be as follows:
1)若设备网络制式为支持多网络制式，无论设备业务类型为固定类业务(如温度、湿度、压力传感器等传输业务)还是移动类业务(如智能标签、作业人员智能头盔等传输业务)，先采用第一深度Q网络，获得选择连接的网络，即切换后的网络，然后采用第二深度Q网络，获得选择连接网络的接入点，即切换后网络的接入点；1) If the device network standard supports multiple network standards, no matter whether the device business type is fixed business (such as temperature, humidity, pressure sensor, etc. transmission business) or mobile business (such as smart label, operator smart helmet, etc. transmission business), First use the first depth Q network to obtain the network selected for connection, that is, the network after switching, and then use the second depth Q network to obtain the access point of the selected connection network, that is, the access point of the network after switching;
2)若设备网络制式为单网络制式且设备业务类型为固定类业务，那么该设备无法进行网络切换；若设备网络制式为单网络制式且设备业务类型为非固定类业务(即移动类业务)，直接采用第二深度Q网络，获得选择连接的网络接入点，即切换后的网络接入点。2) If the device network standard is a single network standard and the device service type is a fixed service, then the device cannot perform network switching; if the device network standard is a single network standard and the device service type is a non-fixed service (ie mobile service) , directly adopting the second deep Q network to obtain the network access point selected for connection, that is, the switched network access point.
上述第一深度Q网络只负责垂直切换(即网络间切换)，第二深度Q网络 只负责接入点切换，第一深度Q网络和第二深度Q网络采用同样的深度Q网络，均包括现实Q网络和目标Q网络，其输入也均为网络环境状态参数，但是由于两者的目的不同，因此输入的参数和网络切换奖励函数不同。The above-mentioned first deep Q-network is only responsible for vertical switching (i.e. inter-network switching), and the second deep Q-network is only responsible for access point switching. The first deep Q-network and the second deep-Q network use the same deep Q-network, including real The Q-network and the target Q-network also have network environment state parameters as their inputs, but because the purposes of the two are different, the input parameters and network switching reward functions are different.
将现实Q网络与无线网络环境进行交互，即将网络环境状态参输入现实Q网络，其中，输入第一深度Q网络的网络环境状态参数可以为：The actual Q network is interacted with the wireless network environment, that is, the network environment state parameters are input into the actual Q network, wherein, the network environment state parameters input into the first deep Q network can be:
A、环境中各无线网络的带宽、时延、误码率和抖动；A. The bandwidth, delay, bit error rate and jitter of each wireless network in the environment;
B、设备对网络带宽、时延、误码率和抖动的需求矩阵。B. The device's demand matrix for network bandwidth, delay, bit error rate and jitter.
假设环境中共有N个异构无线网络，如WiFi、LoRa、5G等，上述参数用公式可表示为：Assuming that there are N heterogeneous wireless networks in the environment, such as WiFi, LoRa, 5G, etc., the above parameters can be expressed as:
S 1＝{B 1,τ 1,e 1,J 1,B 2,τ 2,e 2,J 2,...,B N,τ N,e N,J N,X} S 1 ＝{B 1 ,τ 1 ,e 1 ,J 1 ,B 2 ,τ 2 ,e 2 ,J 2 ,...,B N ,τ N ,e N ,J N ,X}
其中，S 1为输入第一深度Q网络的网络环境状态参数集，B n为无线网络n所提供的带宽，τ n为无线网络n所提供的时延，e n为无线网络n所提供的误码率，J n为无线网络n所提供的抖动，n∈[1,N]，X为设备对网络带宽、时延、误码率和抖动的需求矩阵。 Among them, S 1 is the network environment state parameter set input to the first depth Q network, B n is the bandwidth provided by wireless network n, τ n is the time delay provided by wireless network n, e n is the time delay provided by wireless network n Bit error rate, J n is the jitter provided by wireless network n, n∈[1,N], X is the demand matrix of equipment for network bandwidth, delay, bit error rate and jitter.
输入第二深度Q网络的网络环境状态参数可以为：The network environment state parameter input into the second depth Q network can be:
A、网络中各接入点的带宽、时延、误码率和抖动；A. The bandwidth, delay, bit error rate and jitter of each access point in the network;
B、设备接收各接入点的接收功率；B. The device receives the receiving power of each access point;
C、设备对网络带宽、时延、误码率和抖动的需求矩阵。C. The device's demand matrix for network bandwidth, delay, bit error rate and jitter.
假设网络中的接入点数量为M，上述参数用公式可表示为：Assuming that the number of access points in the network is M, the above parameters can be expressed as:
S 2＝{P 1,B′ 1,τ′ 1,e′ 1,J′ 1,P 1,B′ 2,τ′ 2,e′ 2,J′ 2,...,P M,B′ M,τ′ M,e′ M,J′ M,X} S 2 ＝{P 1 ,B′ 1 ,τ′ 1 ,e′ 1 ,J′ 1 ,P 1 ,B′ 2 ,τ′ 2 ,e′ 2 ,J′ 2 , ...,P M ,B ′ M ,τ′ M ,e′ M ,J′ M ,X}
其中，s 2为输入第二深度Q网络的网络环境状态参数集，P m为接入点m的接收功率，B′ m为接入点m所提供的带宽，τ′ m为接入点m所提供的时延，e′ m为 接入点m所提供的误码率，J′ m为接入点m所提供的抖动。 Among them, s 2 is the network environment state parameter set input into the second depth Q network, P m is the received power of access point m, B′ m is the bandwidth provided by access point m, τ′ m is the access point m The provided delay, e' m is the bit error rate provided by the access point m, and J' m is the jitter provided by the access point m.
将网络环境状态参数输入现实Q网络后得到Q至，使用ε-greedy方法选择动作，其中，对于第一深度Q网络，动作则是选择连接的网络，对于第二深度Q网络，动作则是选择连接的网络接入点，可采用以下公式表示：Input the network environment state parameters into the actual Q network to get Q to, use the ε-greedy method to select the action, where, for the first deep Q network, the action is to select the connected network, and for the second deep Q network, the action is to select The connected network access point can be expressed by the following formula:
Figure PCTCN2022089236-appb-000019
Figure PCTCN2022089236-appb-000019
Figure PCTCN2022089236-appb-000020
Figure PCTCN2022089236-appb-000020
其中，a 1、a 2分别为网络切换动作和网络接入点切换动作，θ 1、θ 1分别为第一深度Q网络和第二深度Q网络的参数，α为生成的一个0～1随机数，ε为探索的概率。 Among them, a 1 and a 2 are network switching actions and network access point switching actions respectively, θ 1 and θ 1 are the parameters of the first deep Q network and the second deep Q network respectively, and α is a generated random number, ε is the probability of exploration.
通过执行动作，环境将返回下一时刻的状态S′ 1或S′ 2、以及设备在执行网络切换动作的奖励。 By executing the action, the environment will return to the state S' 1 or S' 2 at the next moment, and the reward for the device performing the network switching action.
相对于第一深度Q网络，网络切换奖励函数可用以下公式表示：Compared with the first deep Q network, the network switching reward function can be expressed by the following formula:
Figure PCTCN2022089236-appb-000021
Figure PCTCN2022089236-appb-000021
Figure PCTCN2022089236-appb-000022
Figure PCTCN2022089236-appb-000022
Figure PCTCN2022089236-appb-000023
Figure PCTCN2022089236-appb-000023
Figure PCTCN2022089236-appb-000024
Figure PCTCN2022089236-appb-000024
Figure PCTCN2022089236-appb-000025
Figure PCTCN2022089236-appb-000025
其中，r 1为第一深度Q网络中网络切换奖励，
Figure PCTCN2022089236-appb-000026
为网络切换时带宽的网络 选择权重，
Figure PCTCN2022089236-appb-000027
为网络切换时时延的网络选择权重，
Figure PCTCN2022089236-appb-000028
为网络切换时误码率的网络选择权重，
Figure PCTCN2022089236-appb-000029
为网络切换时抖动的网络选择权重，f 1B(S 1,n)为网络切换时带宽的网络选择收益函数，f 1τ(S 1,n)为网络切换时时延的网络选择收益函数，f 1e(S 1,n)为网络切换时误码率的网络选择收益函数，f 1J(S 1,n)为网络切换时抖动的网络选择收益函数，qB为无线网络n需求的带宽，qτ为无线网络n需求的时延，qe为无线网络n需求的误码率，qJ为无线网络n需求的抖动。 Among them, r 1 is the network switching reward in the first deep Q network,
Figure PCTCN2022089236-appb-000026
Select weights for networks with bandwidth when switching between networks,
Figure PCTCN2022089236-appb-000027
Select weights for networks with delays in network switching,
Figure PCTCN2022089236-appb-000028
Select the weight for the network for the bit error rate when the network is switched,
Figure PCTCN2022089236-appb-000029
The network selection weight for the jitter during network switching, f 1B (S 1 ,n) is the network selection revenue function of bandwidth during network switching, f 1τ (S 1 ,n) is the network selection revenue function of network switching delay, f 1e (S 1 ,n) is the network selection revenue function of bit error rate during network switching, f 1J (S 1 ,n) is the network selection revenue function of jitter during network switching, qB is the bandwidth required by wireless network n, and qτ is the wireless The delay required by network n, qe is the bit error rate required by wireless network n, and qJ is the jitter required by wireless network n.
相对于第二深度Q网络，网络切换奖励函数可用以下公式表示：Compared to the second deep Q network, the network switching reward function can be expressed by the following formula:
Figure PCTCN2022089236-appb-000030
Figure PCTCN2022089236-appb-000030
Figure PCTCN2022089236-appb-000031
Figure PCTCN2022089236-appb-000031
Figure PCTCN2022089236-appb-000032
Figure PCTCN2022089236-appb-000032
Figure PCTCN2022089236-appb-000033
Figure PCTCN2022089236-appb-000033
Figure PCTCN2022089236-appb-000034
Figure PCTCN2022089236-appb-000034
其中，r 2为第二深度Q网络中网络切换奖励，
Figure PCTCN2022089236-appb-000035
为选择接入点时带宽的网络选择权重，
Figure PCTCN2022089236-appb-000036
为选择接入点时时延的网络选择权重，
Figure PCTCN2022089236-appb-000037
为选择接入点时的误码率的网络选择权重，
Figure PCTCN2022089236-appb-000038
为选择接入点时的抖动的网络选择权重，f 2B(S 2,m)为选择接入点时带宽的网络选择收益函数，f 2τ(S 2,m)为选择接入点时时延的网络选择收益函数，f 2e(S 2,m)为选择接入点时误码率的网络选择收益函数，f 2J(S 2,m)为选择接入点时抖动的网络选择收益函数，P th为接收功率的灵敏度，qB′为接入点m需求的网络带宽，qτ′为接入点m需求的网络时延，qe′为接入点 m需求的网络误码率，qJ′为接入点m需求的网络抖动。 Among them, r 2 is the network switching reward in the second deep Q network,
Figure PCTCN2022089236-appb-000035
Select weights for networks with bandwidth when selecting access points,
Figure PCTCN2022089236-appb-000036
Select weights for networks with delays when selecting access points,
Figure PCTCN2022089236-appb-000037
Select weights for the network for bit error rates when selecting access points,
Figure PCTCN2022089236-appb-000038
It is the network selection weight of the jitter when selecting the access point, f 2B (S 2 ,m) is the network selection benefit function of the bandwidth when selecting the access point, and f 2τ (S 2 ,m) is the time delay when selecting the access point Network selection revenue function, f 2e (S 2 ,m) is the network selection revenue function of bit error rate when selecting an access point, f 2J (S 2 ,m) is the network selection revenue function of jitter when selecting an access point, P th is the sensitivity of receiving power, qB' is the network bandwidth required by access point m, qτ' is the network delay required by access point m, qe' is the network bit error rate required by access point m, and qJ' is the network bandwidth required by access point m. Network jitter required by entry point m.
上述第一深度Q网络的四维数据(S 1,a 1,r 1,S′ 1)或第二深度Q网络的四维数据(S 2,a 2,r 2,S′ 2)会存储在经验池中，通过经验池中的数据进行深度Q网络训练。 The above four-dimensional data (S 1 , a 1 , r 1 , S′ 1 ) of the first deep Q network or the four-dimensional data (S 2 , a 2 , r 2 , S′ 2 ) of the second deep Q network will be stored in the empirical In the pool, the deep Q network is trained through the data in the experience pool.
将四维数据中的网络环境状态参数(S 1、S 2)输入现实Q网络，得到现实Q值，将四维数据中的下一时刻的状态(S′ 1、S′ 2)输入目标Q网络，得到下一状态的目标Q值，选择使目标Q值最大的切换动作作为下一状态的动作： Input the network environment state parameters (S 1 , S 2 ) in the four-dimensional data into the actual Q network to obtain the real Q value, and input the state (S′ 1 , S′ 2 ) in the four-dimensional data into the target Q network, The target Q value of the next state is obtained, and the switching action that maximizes the target Q value is selected as the action of the next state:
Figure PCTCN2022089236-appb-000039
Figure PCTCN2022089236-appb-000039
a′∈{a′ 1,a′ 2} a′∈{a′ 1 ,a′ 2 }
a∈{a 1,a 2} a∈{a 1 ,a 2 }
S′∈{S′ 1,S′ 2} S′∈{S′ 1 ,S′ 2 }
Figure PCTCN2022089236-appb-000040
Figure PCTCN2022089236-appb-000040
其中，a′表示下一状态的切换动作，θ -表示目标Q网络的网络参数，下标1表示对应第一深度Q网络，2表示对应第二深度Q网络。 Among them, a' indicates the switching action of the next state, θ - indicates the network parameters of the target Q network, the subscript 1 indicates that it corresponds to the first deep Q network, and 2 indicates that it corresponds to the second deep Q network.
获得下一状态动作后，目标Q值可以更新为：After obtaining the next state action, the target Q value can be updated as:
Q′＝r+γQ(S′,a′；θ)Q'=r+γQ(S',a'; θ)
r∈{r 1,r 2} r∈{r 1 ,r 2 }
θ∈{θ 1,θ 2} θ∈{θ 1 ,θ 2 }
其中，γ表示折扣因子，r表示奖励。Among them, γ represents the discount factor and r represents the reward.
利用更新后的目标Q值可以计算损失函数，如图3所示，公式可以表示为：The loss function can be calculated using the updated target Q value, as shown in Figure 3, the formula can be expressed as:
Loss(θ)＝E[r+γQ(S′,a′；θ)-Q(S,a；θ)] 2 Loss(θ)=E[r+γQ(S′,a′;θ)-Q(S,a;θ)] 2
其中，θ为深度Q网络参数，E为取期望操作。Among them, θ is the deep Q network parameter, and E is the expected operation.
通过损失函数进行反向传播更新现实Q网络的网络参数，具体可以为，每隔一定的步数，将当前现实Q网络的网络参数复制给目标Q网络。The network parameters of the real Q network are updated through the backpropagation of the loss function. Specifically, the network parameters of the current real Q network are copied to the target Q network every certain number of steps.
分别使用多网络制式设备和单网络制式设备在某一环境下的网络参数训练 第一深度Q网络和第二深度Q网络，直到网络结果收敛，将训练好的网络应用于异构无线网络垂直切换。Use the network parameters of multi-network standard equipment and single-network standard equipment in a certain environment to train the first deep Q network and the second deep Q network until the network results converge, and apply the trained network to vertical switching of heterogeneous wireless networks .
上述方法可实施在图4所示的异构无线网络场景中，假设该场景为智慧工厂、变电站、地下管廊或大型运动场等，先构建并训练RBF神经网络与深度Q网络，将训练好的网络置入具有计算能力的设备中。每隔一个时间段，RBF神经网络将根据实时的设备状态信息判断设备当前是否支持多网络制式及所传输的业务类型，并通过图2中的方法选择相应的深度Q网络进行切换动作判断。The above method can be implemented in the heterogeneous wireless network scenario shown in Figure 4. Assuming that the scenario is a smart factory, substation, underground pipe corridor or large sports field, etc., first construct and train the RBF neural network and deep Q network, and then the trained The network is built into devices with computing power. Every time period, the RBF neural network will judge whether the device currently supports multi-network standards and the type of business transmitted according to the real-time device status information, and select the corresponding deep Q network to judge the switching action through the method in Figure 2.
上述方法面向无线传感网中的不同设备和不同业务，结合了RBF神经网络和深度Q网络，基于RBF神经网络判断设备业务类型和设备网络制式，从而确定切换所使用的深度Q网络，其中第一深度Q网络负责切换不同的网络制式，第二深度Q网络负责切换同一种网络下的不同接入点。两种深度Q网络均通过网络状态参数及构造的奖励函数进行训练，将当前异构网络或网络接入点的环境状态作为深度Q网络输入，对网络切换进行判决，获得最优网络及接入点，避免了因传统切换算法切换指标单一而引发的误切换，减少设备切换次数，切换更加合理。该方法能够实现不同网络的精准有效网络切换，提高用户的服务质量。The above method is oriented to different devices and different services in the wireless sensor network, combines the RBF neural network and the deep Q network, and judges the device service type and device network standard based on the RBF neural network, so as to determine the deep Q network used for switching. A deep Q network is responsible for switching different network standards, and a second deep Q network is responsible for switching different access points under the same network. Both deep Q-networks are trained through the network state parameters and constructed reward functions, and the current heterogeneous network or the environmental state of the network access point is used as the input of the deep Q-network to judge the network switching and obtain the optimal network and access points. point, avoiding the wrong switching caused by the single switching index of the traditional switching algorithm, reducing the number of equipment switching, and switching more reasonable. The method can realize accurate and effective network switching of different networks, and improve the service quality of users.
基于同样的技术方案，本发明还公开了上述方法相应的软件系统，即一种自适应网络切换系统，包括：Based on the same technical solution, the present invention also discloses a software system corresponding to the above method, that is, an adaptive network switching system, including:
RBF神经网络模块：将网络环境下设备的状态信息输入预先训练的RBF神经网络，获得设备业务类型和设备网络制式；RBF neural network module: Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
第一切换模块：若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络，获得选择连接的网络，采用预先训练的第二深度Q网络，获得选 择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换；The first switching module: if the network standard of the device supports multi-network standard, the pre-trained first deep Q network is used to obtain the network to be selected for connection, and the pre-trained second deep Q network is used to obtain the access point of the selected network, Perform network switching according to the selected network and the access point selected to connect to the network;
第二切换模块：若设备网络制式为单网络制式且设备业务类型为非固定类业务，采用预先训练的第二深度Q网络，获得选择连接的网络接入点，根据选择连接的网络接入点，进行网络切换。The second switching module: if the device network standard is a single network standard and the device business type is non-fixed business, use the pre-trained second depth Q network to obtain the selected network access point, and select the connected network access point to perform network switching.
基于同样的技术方案，本发明还公开了一种存储一个或多个程序的计算机可读存储介质，所述一个或多个程序包括指令，所述指令当由计算设备执行时，使得所述计算设备执行自适应网络切换方法。Based on the same technical solution, the present invention also discloses a computer-readable storage medium that stores one or more programs, and the one or more programs include instructions that, when executed by a computing device, cause the computing The device implements an adaptive network switching method.
基于同样的技术方案，本发明还公开了一种计算设备，包括一个或多个处理器、一个或多个存储器以及一个或多个程序，其中一个或多个程序存储在所述一个或多个存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序包括用于执行自适应网络切换方法的指令。Based on the same technical solution, the present invention also discloses a computing device, including one or more processors, one or more memories, and one or more programs, wherein one or more programs are stored in the one or more stored in the memory and configured to be executed by the one or more processors, the one or more programs include instructions for executing the adaptive network handover method.
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机 或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.
以上仅为本发明的实施例而已，并不用于限制本发明，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均包含在申请待批的本发明的权利要求范围之内。The above is only an embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the pending application of the present invention. within the scope of the claims.

Claims (10)

  1. 一种自适应网络切换方法，其特征在于，包括：A method for adaptive network switching, characterized in that it comprises:
    将网络环境下设备的状态信息输入预先训练的RBF神经网络，获得设备业务类型和设备网络制式；Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
    若设备网络制式为支持多网络制式，将网络环境状态参数作为输入，采用预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换；If the device network standard supports multiple network standards, the network environment state parameters are used as input, and the pre-trained first deep Q network and second deep Q network are used to obtain the selected network and the access point of the selected network, according to the selection Connected network and select the access point connected to the network to switch the network;
    若设备网络制式为单网络制式且设备业务类型为非固定类业务，将网络环境状态参数作为输入，采用预先训练的第二深度Q网络，获得选择连接的网络接入点，根据选择连接的网络接入点，进行网络切换。If the network standard of the device is a single network standard and the business type of the device is non-fixed business, the network environment status parameters are used as input, and the pre-trained second depth Q network is used to obtain the network access point for the selected connection. Access point for network switching.
  2. 根据权利要求1所述的一种自适应网络切换方法，其特征在于，设备的状态信息包括设备接收功率变化值、设备接收时延变化值和设备固有参数。The adaptive network switching method according to claim 1, wherein the status information of the device includes a change value of device receiving power, a change value of device receiving delay and inherent parameters of the device.
  3. 根据权利要求1所述的一种自适应网络切换方法，其特征在于，若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换，包括：A method for adaptive network switching according to claim 1, wherein, if the network standard of the device supports multiple network standards, the pre-trained first depth Q network and the second depth Q network are used to obtain a network for selective connection And select the access point to connect to the network, and perform network switching according to the selected network and the selected access point to connect to the network, including:
    若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络，获得选择连接的网络，采用预先训练的第二深度Q网络，获得选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换。If the network standard of the device supports multiple network standards, use the pre-trained first depth Q network to obtain the network to be selected for connection, and use the pre-trained second depth Q network to obtain the access point to select the network to connect to. and select the access point connected to the network to switch the network.
  4. 根据权利要求3所述的一种自适应网络切换方法，其特征在于，第一深度Q网络的输入为网络环境状态参数，包括：A kind of adaptive network switching method according to claim 3, is characterized in that, the input of the first depth Q network is network environment state parameter, comprises:
    环境中各无线网络的带宽、时延、误码率和抖动；The bandwidth, delay, bit error rate and jitter of each wireless network in the environment;
    设备对网络带宽、时延、误码率和抖动的需求矩阵。The device's demand matrix for network bandwidth, delay, bit error rate, and jitter.
  5. 根据权利要求4所述的一种自适应网络切换方法，其特征在于，第一深度Q网络中网络切换奖励为：A method for adaptive network switching according to claim 4, wherein the network switching reward in the first deep Q network is:
    Figure PCTCN2022089236-appb-100001
    Figure PCTCN2022089236-appb-100001
    其中，r 1为第一深度Q网络中网络切换奖励，
    Figure PCTCN2022089236-appb-100002
    为网络切换时带宽的网络选择权重，
    Figure PCTCN2022089236-appb-100003
    为网络切换时时延的网络选择权重，
    Figure PCTCN2022089236-appb-100004
    为网络切换时误码率的网络选择权重，
    Figure PCTCN2022089236-appb-100005
    为网络切换时抖动的网络选择权重，f 1B(S 1,n)为网络切换时带宽的网络选择收益函数，f 1τ(S 1,n)为网络切换时时延的网络选择收益函数，f 1e(S 1,n)为网络切换时误码率的网络选择收益函数，f 1J(S 1,n)为网络切换时抖动的网络选择收益函数，S 1为输入第一深度Q网络的网络环境状态参数集。     Among them, r 1 is the network switching reward in the first deep Q network,
    Figure PCTCN2022089236-appb-100002
    Select weights for networks with bandwidth when switching between networks,
    Figure PCTCN2022089236-appb-100003
    Select weights for networks with delays in network switching,
    Figure PCTCN2022089236-appb-100004
    Select the weight for the network for the bit error rate when the network is switched,
    Figure PCTCN2022089236-appb-100005
    The network selection weight for the jitter during network switching, f 1B (S 1 ,n) is the network selection revenue function of bandwidth during network switching, f 1τ (S 1 ,n) is the network selection revenue function of network switching delay, f 1e (S 1 ,n) is the network selection benefit function of bit error rate during network switching, f 1J (S 1 ,n) is the network selection benefit function of jitter during network switching, and S 1 is the network environment input to the first deep Q network State parameter set.
    Figure PCTCN2022089236-appb-100006
    Figure PCTCN2022089236-appb-100006
    Figure PCTCN2022089236-appb-100007
    Figure PCTCN2022089236-appb-100007
    Figure PCTCN2022089236-appb-100008
    Figure PCTCN2022089236-appb-100008
    Figure PCTCN2022089236-appb-100009
    Figure PCTCN2022089236-appb-100009
    其中，B n为无线网络n所提供的带宽，τ n为无线网络n所提供的时延，e n为无线网络n所提供的误码率，J n为无线网络n所提供的抖动，qB为无线网络n需求的带宽，qτ为无线网络n需求的时延，qe为无线网络n需求的误码率，qJ为无线网络n需求的抖动。 Among them, B n is the bandwidth provided by wireless network n, τ n is the delay provided by wireless network n, e n is the bit error rate provided by wireless network n, J n is the jitter provided by wireless network n, qB is the bandwidth required by wireless network n, qτ is the delay required by wireless network n, qe is the bit error rate required by wireless network n, and qJ is the jitter required by wireless network n.
  6. 根据权利要求3所述的一种自适应网络切换方法，其特征在于，第二深 度Q网络的输入为网络环境状态参数，包括：A kind of adaptive network switching method according to claim 3, is characterized in that, the input of the second depth Q network is network environment state parameter, comprises:
    网络中各接入点的带宽、时延、误码率和抖动；Bandwidth, delay, bit error rate and jitter of each access point in the network;
    设备接收各接入点的接收功率；The device receives the receiving power of each access point;
    设备对网络带宽、时延、误码率和抖动的需求矩阵。The device's demand matrix for network bandwidth, delay, bit error rate, and jitter.
  7. 根据权利要求6所述的一种自适应网络切换方法，其特征在于，第二深度Q网络中网络切换奖励为：A kind of adaptive network switching method according to claim 6, is characterized in that, the network switching award in the second depth Q network is:
    Figure PCTCN2022089236-appb-100010
    Figure PCTCN2022089236-appb-100010
    其中，r 2为第二深度Q网络中网络切换奖励，
    Figure PCTCN2022089236-appb-100011
    为选择接入点时带宽的网络选择权重，
    Figure PCTCN2022089236-appb-100012
    为选择接入点时时延的网络选择权重，
    Figure PCTCN2022089236-appb-100013
    为选择接入点时误码率的网络选择权重，
    Figure PCTCN2022089236-appb-100014
    为选择接入点时抖动的网络选择权重，f 2B(S 2,m)为选择接入点时带宽的网络选择收益函数，f 2τ(S 2,m)为选择接入点时时延的网络选择收益函数，f 2e(S 2,m)为选择接入点时误码率的网络选择收益函数，f 2J(S 2,m)为选择接入点时抖动的网络选择收益函数，S 2为输入第二深度Q网络的网络环境状态参数集，P m为接入点m的接收功率，P th为接收功率的灵敏度     Among them, r 2 is the network switching reward in the second deep Q network,
    Figure PCTCN2022089236-appb-100011
    Select weights for networks with bandwidth when selecting access points,
    Figure PCTCN2022089236-appb-100012
    Select weights for networks with delays when selecting access points,
    Figure PCTCN2022089236-appb-100013
    Select the weight for the network for the bit error rate when selecting an access point,
    Figure PCTCN2022089236-appb-100014
    Select the weight for the jitter network when selecting the access point, f 2B (S 2 ,m) is the network selection revenue function of the bandwidth when selecting the access point, and f 2τ (S 2 ,m) is the network delay when selecting the access point Select the income function, f 2e (S 2 ,m) is the network selection income function of bit error rate when selecting the access point, f 2J (S 2 ,m) is the network selection income function of jitter when selecting the access point, S 2 is the network environment state parameter set input to the second deep Q network, P m is the received power of access point m, and P th is the sensitivity of the received power
    Figure PCTCN2022089236-appb-100015
    Figure PCTCN2022089236-appb-100015
    Figure PCTCN2022089236-appb-100016
    Figure PCTCN2022089236-appb-100016
    Figure PCTCN2022089236-appb-100017
    Figure PCTCN2022089236-appb-100017
    Figure PCTCN2022089236-appb-100018
    Figure PCTCN2022089236-appb-100018
    其中，B′ m为接入点m所提供的带宽，τ′ m为接入点m所提供的时延，e′ m为 接入点m所提供的误码率，J′ m为接入点m所提供的抖动，qB′为接入点m需求的网络带宽，qτ′为接入点m需求的网络时延，qe′为接入点m需求的网络误码率，qJ′为接入点m需求的网络抖动。 Among them, B′ m is the bandwidth provided by the access point m, τ′ m is the delay provided by the access point m, e′ m is the bit error rate provided by the access point m, and J′ m is the access point The jitter provided by point m, qB' is the network bandwidth required by access point m, qτ' is the network delay required by access point m, qe' is the network bit error rate required by access point m, and qJ' is the network bandwidth required by access point m. Network jitter required by entry point m.
  8. 一种自适应网络切换系统，其特征在于，包括：An adaptive network switching system is characterized in that it includes:
    RBF神经网络模块：将网络环境下设备的状态信息输入预先训练的RBF神经网络，获得设备业务类型和设备网络制式；RBF neural network module: Input the status information of the equipment in the network environment into the pre-trained RBF neural network to obtain the equipment business type and equipment network standard;
    第一切换模块：若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络和第二深度Q网络，获得选择连接的网络和选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换；The first switching module: if the network standard of the device supports multiple network standards, the pre-trained first deep Q network and the second deep Q network are used to obtain the selected network and the access point of the selected network, according to the selected network And select the access point connected to the network to switch the network;
    第二切换模块：若设备网络制式为单网络制式且设备业务类型为非固定类业务，采用预先训练的第二深度Q网络，获得选择连接的网络接入点，根据选择连接的网络接入点，进行网络切换。The second switching module: if the device network standard is a single network standard and the device business type is non-fixed business, use the pre-trained second depth Q network to obtain the selected network access point, and select the connected network access point to perform network switching.
  9. 根据权利要求8所述的一种自适应网络切换系统，其特征在于，第一切换模块：若设备网络制式为支持多网络制式，采用预先训练的第一深度Q网络，获得选择连接的网络，采用预先训练的第二深度Q网络，获得选择连接网络的接入点，根据选择连接的网络和选择连接网络的接入点，进行网络切换。An adaptive network switching system according to claim 8, characterized in that, the first switching module: if the device network standard supports multi-network standard, the pre-trained first depth Q network is used to obtain the network for selective connection, The pre-trained second deep Q network is used to obtain the access point of the selected network, and the network switching is performed according to the selected network and the selected access point of the network.
  10. 一种存储一个或多个程序的计算机可读存储介质，其特征在于：所述一个或多个程序包括指令，所述指令当由计算设备执行时，使得所述计算设备执行根据权利要求1至7所述的方法中的任一方法。A computer-readable storage medium storing one or more programs, wherein the one or more programs comprise instructions which, when executed by a computing device, cause the computing device to perform the Any one of the methods described in 7.
PCT/CN2022/089236 2021-11-10 2022-04-26 Adaptive network switching method and system, and storage medium WO2023082554A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020227030640A KR20230070405A (en) 2021-11-10 2022-04-26 Adaptive network handover method, system and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111325359.2 2021-11-10
CN202111325359.2A CN114125962B (en) 2021-11-10 Self-adaptive network switching method, system and storage medium

Publications (1)

Publication Number Publication Date
WO2023082554A1 true WO2023082554A1 (en) 2023-05-19

Family

ID=80377909

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/089236 WO2023082554A1 (en) 2021-11-10 2022-04-26 Adaptive network switching method and system, and storage medium

Country Status (2)

Country Link
KR (1) KR20230070405A (en)
WO (1) WO2023082554A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107979859A (en) * 2016-10-21 2018-05-01 中兴通讯股份有限公司 A kind of method for switching network, device, terminal and the webserver
CN108235390A (en) * 2017-12-01 2018-06-29 吉林大学 Vertical handoff method based on Bayesian decision in a kind of heterogeneous wireless network
US20210241090A1 (en) * 2020-01-31 2021-08-05 At&T Intellectual Property I, L.P. Radio access network control with deep reinforcement learning
CN113490248A (en) * 2021-07-27 2021-10-08 重庆邮电大学 Multi-mode terminal switching method and device
CN114125962A (en) * 2021-11-10 2022-03-01 国网江苏省电力有限公司电力科学研究院 Self-adaptive network switching method, system and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107979859A (en) * 2016-10-21 2018-05-01 中兴通讯股份有限公司 A kind of method for switching network, device, terminal and the webserver
CN108235390A (en) * 2017-12-01 2018-06-29 吉林大学 Vertical handoff method based on Bayesian decision in a kind of heterogeneous wireless network
US20210241090A1 (en) * 2020-01-31 2021-08-05 At&T Intellectual Property I, L.P. Radio access network control with deep reinforcement learning
CN113490248A (en) * 2021-07-27 2021-10-08 重庆邮电大学 Multi-mode terminal switching method and device
CN114125962A (en) * 2021-11-10 2022-03-01 国网江苏省电力有限公司电力科学研究院 Self-adaptive network switching method, system and storage medium

Also Published As

Publication number Publication date
KR20230070405A (en) 2023-05-23
CN114125962A (en) 2022-03-01

Similar Documents

Publication Publication Date Title
Chen et al. Federated echo state learning for minimizing breaks in presence in wireless virtual reality networks
CN107113668A (en) Experience type handover management quality
Scanzio et al. Predicting Wi‐Fi link quality through artificial neural networks
Chua et al. Resource allocation for mobile metaverse with the Internet of Vehicles over 6G wireless communications: A deep reinforcement learning approach
Narimani et al. QoS-aware resource allocation and fault tolerant operation in hybrid SDN using stochastic network calculus
Yi et al. DMADRL: A distributed multi-agent deep reinforcement learning algorithm for cognitive offloading in dynamic MEC networks
WO2023082554A1 (en) Adaptive network switching method and system, and storage medium
Zhou et al. A network selection scheme based on the analytic hierarchy process for marine internet
CN114125962B (en) Self-adaptive network switching method, system and storage medium
Gao et al. An Efficient Approximation for Nakagami‐m Quantile Function Based on Generalized Opposition‐Based Quantum Salp Swarm Algorithm
Vankayala et al. Deep-learning based proactive handover for 5G/6G mobile networks using wireless information
WO2023186091A1 (en) Sample determination method, apparatus and device
TW202224472A (en) Beam selection method based on neural network and management server
WO2024032695A1 (en) Csi prediction processing method and apparatus, communication device, and readable storage medium
WO2024032694A1 (en) Csi prediction processing method and apparatus, communication device, and readable storage medium
WO2023186099A1 (en) Information feedback method and apparatus, and device
EP4373159A1 (en) Ai/ml model functionality in handover scenarios
WO2024083004A1 (en) Ai model configuration method, terminal, and network side device
WO2023134653A1 (en) Communication network prediction method, and terminal and network-side device
Wei et al. Cooperative caching algorithm for mobile edge networks based on multi-agent meta reinforcement learning
WO2023103911A1 (en) Measurement method and apparatus, device, and storage medium
WO2023246343A1 (en) Data processing method and apparatus, and computer device, storage medium and product
Si et al. Edge Caching Strategy Based on User's Long and Short Term Interests
WO2024027576A1 (en) Performance supervision method and apparatus for ai network model, and communication device
WO2024104070A1 (en) Beam report sending method and apparatus, beam report receiving method and apparatus, and communication device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22891366

Country of ref document: EP

Kind code of ref document: A1