WO2025251297A1 - 切换策略确定方法、通信设备、通信系统、存储介质 - Google Patents

切换策略确定方法、通信设备、通信系统、存储介质

Info

Publication number
WO2025251297A1
WO2025251297A1 PCT/CN2024/098126 CN2024098126W WO2025251297A1 WO 2025251297 A1 WO2025251297 A1 WO 2025251297A1 CN 2024098126 W CN2024098126 W CN 2024098126W WO 2025251297 A1 WO2025251297 A1 WO 2025251297A1
Authority
WO
WIPO (PCT)
Prior art keywords
satellite
handover
strategy
model
switching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/098126
Other languages
English (en)
French (fr)
Inventor
来庆宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Xiaomi Mobile Software Co Ltd
Original Assignee
Beijing Xiaomi Mobile Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Xiaomi Mobile Software Co Ltd filed Critical Beijing Xiaomi Mobile Software Co Ltd
Priority to PCT/CN2024/098126 priority Critical patent/WO2025251297A1/zh
Priority to CN202480039750.1A priority patent/CN121511630A/zh
Publication of WO2025251297A1 publication Critical patent/WO2025251297A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W36/00—Hand-off or reselection arrangements

Definitions

  • This disclosure relates to the field of communication technology, and in particular to a method for determining handover strategies, communication equipment, communication systems, and storage media.
  • terminals or ground stations connect to satellites.
  • the terminal or ground station typically needs to perform satellite handover (HO), for example, when the satellite to which the terminal or ground station is connected moves to an invisible position, the terminal or ground station can switch from the currently connected satellite to another visible satellite to ensure the continuity of satellite service.
  • HO satellite handover
  • different handover strategies are fixedly set for different types of satellites.
  • This disclosure proposes a method for determining a switching strategy, communication equipment, communication system, and storage medium.
  • a handover strategy determination method executed by a first device, the method comprising:
  • the first parameter is used to indicate the network environment parameters of the non-terrestrial network NTN where the second device is located; the second device is a device that needs to perform satellite handover.
  • the first parameter is input into the first model to obtain the switching strategy output by the first model; wherein the first model is used to perform reinforcement learning or deep reinforcement learning, and the switching strategy is used by the second device to perform satellite switching.
  • a first device comprising:
  • the processing module is used to determine a first parameter; the first parameter is used to indicate the network environment parameters of the non-terrestrial network NTN where the second device is located; the second device is a device that needs to perform satellite handover.
  • the processing module is further configured to input the first parameter into the first model to obtain the switching strategy output by the first model; wherein the first model is used to perform reinforcement learning or deep reinforcement learning, and the switching strategy is used by the second device to perform satellite switching.
  • a communication device comprising:
  • One or more processors are One or more processors;
  • the processor is configured to invoke instructions to cause the communication device to execute the method described in the first aspect.
  • a communication system includes at least one of a satellite, a second device, and a third device, wherein the satellite, the second device, or the third device is configured to implement the method described in the first aspect.
  • a storage medium that stores instructions, characterized in that, when the instructions are executed on a communication device, the communication device performs the method as described in the first aspect.
  • embodiments of this disclosure provide a program product that, when executed by a communication device, causes the communication device to perform the method described in the first aspect.
  • embodiments of this disclosure provide a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect.
  • inventions of this disclosure provide a chip or chip system.
  • the chip or chip system includes processing circuitry configured to perform the method according to the first aspect described above.
  • Figure 1 is a schematic diagram of the architecture of some communication systems provided in the embodiments of this disclosure.
  • Figure 2A is a flowchart illustrating a method for determining a switching strategy provided in another embodiment of this disclosure
  • Figure 2B is a flowchart illustrating the nearest satellite handover strategy according to an embodiment of the present disclosure
  • Figure 2C is a flowchart illustrating a satellite visibility-based handover strategy according to an embodiment of the present disclosure
  • Figure 2D is a formula for calculating the Q value according to an embodiment of the present disclosure.
  • Figure 2E is a formula for calculating the loss function according to an embodiment of the present disclosure.
  • Figure 2F is an interactive schematic diagram of a switching strategy determination method according to an embodiment of the present disclosure.
  • Figure 2G is an interactive schematic diagram of a switching strategy determination method according to an embodiment of the present disclosure.
  • Figure 2H is an interactive schematic diagram of a switching strategy determination method according to an embodiment of the present disclosure
  • Figure 3 is an interactive schematic diagram of a switching strategy determination method according to an embodiment of the present disclosure
  • Figure 4A is an interactive schematic diagram of a switching strategy determination method according to an embodiment of the present disclosure
  • Figure 4B is a schematic diagram of the structure of a deep learning method according to an embodiment of the present disclosure.
  • Figure 4C is a formula for calculating the cumulative function of random rewards according to an embodiment of the present disclosure.
  • Figure 4D is a schematic diagram illustrating a Q-learning process according to an embodiment of the present disclosure.
  • Figure 4E is a schematic diagram of a deep Q-network neural network according to an embodiment of the present disclosure.
  • Figure 5 is a schematic diagram of the structure of the first device proposed in an embodiment of this disclosure.
  • Figure 6A is a schematic diagram of the structure of a communication device provided in an embodiment of this disclosure.
  • Figure 6B is a schematic diagram of the structure of a chip provided in an embodiment of this disclosure.
  • This disclosure provides a method for determining a switching strategy, a communication device, a communication system, and a storage medium.
  • embodiments of this disclosure propose a handover strategy determination method, executed by a first device, the method comprising:
  • the first parameter is used to indicate the network environment parameters of the non-terrestrial network NTN where the second device is located; the second device is a device that needs to perform satellite handover.
  • the first parameter is input into the first model to obtain the switching strategy output by the first model; wherein the first model is used to perform reinforcement learning or deep reinforcement learning, and the switching strategy is used by the second device to perform satellite switching.
  • the first device uses a first model to determine a corresponding handover strategy based on the network environment parameters of the NTN where the second device is located. This ensures that the determined handover strategy is compatible with the network environment of the second device. Therefore, when the second device performs satellite handover, a suitable handover strategy is adopted based on the different network environments in which the second device is located, improving the flexibility of satellite handover, avoiding waste of handover resources, and meeting the real-time requirements of handover services. Furthermore, since the handover strategy in this embodiment is determined based on the network environment of the second device and is independent of the "satellite type," the handover strategy of this method is not limited by the satellite type. This allows for handover between different types of satellites, ensuring the collaborative operation of different types of satellites and guaranteeing communication performance.
  • the switching strategy output by the first model is the switching strategy that achieves the first effect among the alternative switching strategies.
  • the alternative switching strategy includes at least one of the following:
  • the first strategy is: the nearest satellite handover strategy based on RACH-less random access channel;
  • the second strategy is: a nearest satellite handover strategy not based on RACH-less;
  • the third strategy is a switching strategy based on RACH-less and satellite visibility.
  • the fourth strategy is a handover strategy based on satellite visibility but not on RACH-less.
  • the fifth strategy is a handover strategy based on RACH-less and carrier interference-to-noise ratio (CINR).
  • CINR carrier interference-to-noise ratio
  • the sixth strategy is a switching strategy based on CINR but not on RACH-less.
  • the first effect includes at least one of the following:
  • the link spectral efficiency is greater than the first value
  • the switching rate is less than the second value
  • the handover success rate is greater than the third value
  • the switching delay is less than the fourth value
  • the switching interruption time is less than the fifth value
  • the Doppler frequency shift is less than the sixth value.
  • the switching strategy output by the first model is: to achieve the first effect... Choose the best performing alternative switching strategy from the switching strategies; or
  • the switching strategy output by the first model is any one of the alternative switching strategies that achieves the first effect.
  • the first model when the first model outputs a switching strategy based on the network environment parameters of the second device, it outputs "the alternative switching strategy that can achieve the first effect when switching in the current network environment" from the alternative switching strategies. This ensures that the switching strategy output by the first model is not only adapted to the current network environment, but also achieves a better switching effect, thereby improving the switching performance during satellite switching.
  • the network environment parameters of the NTN where the second device is located include at least one of the following:
  • At least one relevant parameter of a first satellite is a visible satellite of the second device;
  • At least one second satellite's relevant parameters the second satellite is a visible satellite of the third satellite that the second device is currently communicating with.
  • the relevant parameters of the first satellite include at least one of the following:
  • the distance between the second device and the first satellite is the distance between the second device and the first satellite
  • the signal strength between the second device and the first satellite is the signal strength between the second device and the first satellite.
  • the relevant parameters of the second satellite include at least one of the following:
  • the signal strength between the third satellite and the second satellite is the signal strength between the third satellite and the second satellite.
  • the first satellite and the second satellite respectively include at least one of the following: a low Earth orbit (LEO) satellite, a medium Earth orbit (MEO) satellite, and a highly elliptical orbit (HEO) satellite.
  • LEO low Earth orbit
  • MEO medium Earth orbit
  • HEO highly elliptical orbit
  • the specific content of the network environment parameters is defined so that the first device can successfully determine these network environment parameters and further determine the handover strategy based on these network environment parameters.
  • the first device includes at least one of any satellite, a second device, and a third device;
  • the second device includes a terminal or ground station; the third device is a ground device different from the second device.
  • the first device is any satellite, and determining the first parameter includes:
  • the first device determines the relevant parameters of at least one second satellite based on the implementation.
  • the first device is a second device, and determining the first parameter includes:
  • the first device determines relevant parameters of at least one first satellite based on the implementation
  • the first device is a third device, and determining the first parameter includes:
  • a method for determining network environment parameters is provided so that the first device can use the method to successfully determine the network environment parameters and further determine the handover strategy based on these network environment parameters.
  • the first device is any satellite, and the method further includes:
  • the second device is instructed to use the handover strategy output by the first model, so that the second device can perform satellite handover based on the handover strategy.
  • the first device is a second device, and the method further includes:
  • Satellite handover is performed based on the handover strategy output by the first model
  • the device reports the switching strategy output by the first model to the third satellite currently communicating with the second device, and receives the switching strategy sent by the third satellite, and performs satellite switching based on the switching strategy sent by the third satellite; wherein the switching strategy sent by the third satellite is the same as or different from the switching strategy output by the first model.
  • the first device is a third device, and the method further includes:
  • the handover strategy output by the first model is reported to the third satellite currently communicating with the second device, so that the third satellite can determine the final handover strategy; wherein the final handover strategy determined by the third satellite is the same as or different from the handover strategy output by the first model.
  • a method for satellite handover based on the handover strategy output by the first model is proposed so that satellite handover can be successfully performed based on the handover strategy output by the first model in the future.
  • the model algorithm of the first model includes at least one of the following algorithms: Q-Learning algorithm, Deep Q-Network (DQN) algorithm, Dual Deep Q-Network (DDQN) algorithm, and Actor-Evaluation (Critic) algorithm.
  • model algorithms for the first model are provided so that the first model can be trained based on these algorithms, ensuring that the first model can accurately output the switching strategy based on network environment parameters, thus guaranteeing the accuracy of the switching strategy determination.
  • inventions of this disclosure provide a first device, comprising:
  • the processing module is used to determine a first parameter; the first parameter is used to indicate the network environment parameters of the non-terrestrial network NTN where the second device is located; the second device is a device that needs to perform satellite handover.
  • the processing module is further configured to input the first parameter into the first model to obtain the switching strategy output by the first model; wherein the first model is used to perform reinforcement learning or deep reinforcement learning, and the switching strategy is used by the second device to perform satellite switching.
  • the switching strategy output by the first model is the switching strategy that achieves the first effect among the alternative switching strategies.
  • the alternative switching strategy includes at least one of the following:
  • the first strategy is: the nearest satellite handover strategy based on RACH-less random access channel;
  • the second strategy is: a nearest satellite handover strategy not based on RACH-less;
  • the third strategy is a switching strategy based on RACH-less and satellite visibility.
  • the fourth strategy is a handover strategy based on satellite visibility but not on RACH-less.
  • the fifth strategy is a handover strategy based on RACH-less and carrier interference-to-noise ratio (CINR).
  • CINR carrier interference-to-noise ratio
  • the sixth strategy is a switching strategy based on CINR but not on RACH-less.
  • the first effect includes at least one of the following:
  • the link spectral efficiency is greater than the first value
  • the switching rate is less than the second value
  • the handover success rate is greater than the third value
  • the switching delay is less than the fourth value
  • the switching interruption time is less than the fifth value
  • the Doppler frequency shift is less than the sixth value.
  • the switching strategy output by the first model is: the best-performing alternative switching strategy among the alternative switching strategies that achieve the first effect; or
  • the switching strategy output by the first model is any one of the alternative switching strategies that achieves the first effect.
  • the network environment parameters of the NTN where the second device is located include at least one of the following:
  • At least one relevant parameter of a first satellite is a visible satellite of the second device;
  • At least one second satellite's relevant parameters the second satellite is a visible satellite of the third satellite that the second device is currently communicating with.
  • the relevant parameters of the first satellite include at least one of the following:
  • the distance between the second device and the first satellite is the distance between the second device and the first satellite
  • the signal strength between the second device and the first satellite is the signal strength between the second device and the first satellite.
  • the relevant parameters of the second satellite include at least one of the following:
  • the signal strength between the third satellite and the second satellite is the signal strength between the third satellite and the second satellite.
  • the first satellite and the second satellite each include at least one of the following: low... LEO satellites, MEO satellites, and HEO satellites.
  • the first device includes at least one of any satellite, a second device, and a third device;
  • the second device includes a terminal or ground station; the third device is a ground device different from the second device.
  • the first device is any satellite, and determining the first parameter includes:
  • the first device determines the relevant parameters of at least one second satellite based on the implementation.
  • the first device is a second device, and determining the first parameter includes:
  • the first device determines relevant parameters of at least one first satellite based on the implementation
  • the first device is a third device, and determining the first parameter includes:
  • the first device is any satellite, and the method further includes:
  • the second device is instructed to use the handover strategy output by the first model, so that the second device can perform satellite handover based on the handover strategy.
  • the first device is a second device, and the method further includes:
  • Satellite handover is performed based on the handover strategy output by the first model
  • the device reports the switching strategy output by the first model to the third satellite currently communicating with the second device, and receives the switching strategy sent by the third satellite, and performs satellite switching based on the switching strategy sent by the third satellite; wherein the switching strategy sent by the third satellite is the same as or different from the switching strategy output by the first model.
  • the first device is a third device, and the method further includes:
  • the handover strategy output by the first model is reported to the third satellite currently communicating with the second device, so that the third satellite can determine the final handover strategy; wherein the final handover strategy determined by the third satellite is the same as or different from the handover strategy output by the first model.
  • the model algorithm of the first model includes at least one of the following algorithms: Q-Learning algorithm, Deep Q-Network (DQN) algorithm, Dual Deep Q-Network (DDQN) algorithm, and Actor-Evaluation (Critic) algorithm.
  • embodiments of this disclosure provide a communication device, which includes: one or more processors; one or more memories for storing instructions; wherein the processors are used to invoke the instructions to cause the communication device to perform the methods described in the first aspect, the optional implementation of the first aspect, the second aspect, and the optional implementation of the second aspect.
  • embodiments of this disclosure propose a communication system comprising at least one of a satellite, a second device, and a third device; wherein the satellite, the second device, or the third device is configured to perform the method described in the first aspect and optional implementations thereof.
  • embodiments of this disclosure provide a storage medium storing instructions that, when executed on a communication device, cause the communication device to perform the method described in the first aspect and its optional implementations.
  • embodiments of this disclosure provide a program product, including a computer program that, when executed by a communication device, implements the method described in the first aspect and its optional implementations.
  • embodiments of this disclosure provide a computer program that, when run on a computer, causes the computer to perform the methods described in the first aspect and optional implementations of the first aspect.
  • embodiments of this disclosure provide a chip or chip system including processing circuitry configured to perform the method described in accordance with the first aspect and optional implementations thereof.
  • This disclosure provides a handover strategy determination method, a communication device, a communication system, and a storage medium.
  • the terms “handover strategy determination method” and “information processing method,” “information sending method,” and “information receiving method” can be used interchangeably; the terms “communication device” and “information processing device,” “information sending device,” and “information receiving device” can be used interchangeably; and the terms “information processing system,” “communication system,” “information sending system,” and “information receiving system” can be used interchangeably.
  • each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined.
  • a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged.
  • the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.
  • multiple refers to two or more.
  • the terms “at least one of”, “at least one of”, “at least one of”, “one or more”, “a plurality of”, “multiple”, etc., may be used interchangeably.
  • the notation "in one case A, in another case B” or “in response to one case A, in response to another case B” may include the following technical solutions depending on the situation: A is executed regardless of B, i.e., A is executed in some embodiments; B is executed regardless of A, i.e., B is executed in some embodiments; A and B are selectively executed, i.e., A and B are selected for execution in some embodiments; A and B are both executed, i.e., A and B are executed in some embodiments. The same applies when there are more branches such as A, B, and C.
  • the descriptive object is a "field,” the ordinal numbers preceding "field” in “first field” and “second field” do not restrict the position or order of the "fields.” "First” and “second” do not restrict whether the "fields” they modify are in the same message, nor do they restrict the order of "first field” and “second field.”
  • the descriptive object is a "level,” the ordinal numbers preceding "level” in “first level” and “second level” do not restrict the priority between “levels.”
  • the number of descriptive objects is not limited by ordinal numbers and can be one or more. For example, in “first device,” the number of "devices" can be one or more.
  • the objects modified by different prefixes can be the same or different.
  • first device and second device can be the same device or different devices, and their types can be the same or different.
  • first information and second information can be the same information or different information, and their content can be the same or different.
  • “including A,” “containing A,” “for indicating A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.
  • the terms “in response to...”, “in response to determining...”, “in the case of...”, “when...”, “if...”, “if...”, etc., can be used interchangeably.
  • the terms “greater than,” “greater than or equal to,” “not less than,” “more than,” “more than or equal to,” “not less than,” “higher than,” “higher than or equal to,” “not lower than,” and “above” can be used interchangeably, as can the terms “less than,” “less than or equal to,” “not greater than,” “less than,” “less than or equal to,” “not more than,” “lower than,” “lower than or equal to,” “not higher than,” and “below”.
  • devices, etc. may be interpreted as physical or virtual, and their names are not limited to those described in the embodiments.
  • the terms “device”, “equipment”, “circuit”, “network element”, “node”, “function”, “unit”, “section”, “system”, “network”, “chip”, “chip system”, “entity”, and “subject” can be used interchangeably.
  • network can be interpreted as devices included in a network (e.g., access network devices, core network devices, etc.).
  • the terms “access network device (AN device),” “radio access network device (RAN device),” “base station (BS),” “radio base station,” “fixed station,” “node,” “access point,” “transmission point (TP),” “reception point (RP),” “transmission/reception point (TRP),” “panel,” “antenna panel,” “antenna array,” “cell,” “macro cell,” “small cell,” “femto cell,” “pico cell,” “sector,” “cell group,” “carrier,” “component carrier,” and “bandwidth part (BWP)” can be used interchangeably.
  • terminal In some embodiments, the terms "terminal”, “terminal device”, “user equipment (UE)”, “user terminal”, “mobile station (MS)”, “mobile terminal (MT)", “subscriber station”, “mobile unit”, “subscriber unit”, “wireless unit”, “remote unit”, “mobile device”, “wireless communication device”, “remote device”, “mobile subscriber station”, “access terminal”, “mobile terminal”, “wireless terminal”, “remote terminal”, “handset”, “user agent”, “mobile client”, and “client” can be used interchangeably.
  • access network devices, core network devices, or network devices can be replaced by terminals.
  • embodiments of this disclosure can also be applied to structures that replace communication between access network devices, core network devices, or network devices and terminals with communication between multiple terminals (e.g., also referred to as device-to-device (D2D), vehicle-to-everything (V2X), etc.).
  • the structure can also be configured such that the terminal has all or part of the functions of the access network device.
  • terms such as "uplink” and “downlink” can be replaced with terms corresponding to communication between terminals (e.g., "sidelink”).
  • uplink channel, downlink channel, etc. can be replaced with sidelink channel
  • uplink link, downlink link, etc. can be replaced with sidelink link.
  • the terminal may be replaced by an access network device, a core network device, or a network device.
  • the access network device, core network device, or network device may also be configured to have all or some of the functions of the terminal.
  • the acquisition of data, information, etc. may comply with the laws and regulations of the country where the location is situated.
  • data, information, etc. may be obtained with the user's consent.
  • each element, each row, or each column in the table of this disclosure can be implemented as an independent embodiment, and any combination of any element, any row, or any column can also be implemented as an independent embodiment.
  • the correspondences shown in the tables of this disclosure can be configured or predefined.
  • the values of the information in each table are merely examples and can be configured to other values; this disclosure is not limiting.
  • it is not necessarily required to configure all the correspondences shown in each table.
  • the correspondences shown in some rows of the tables in this disclosure may not be configured.
  • appropriate modifications and adjustments can be made based on the above tables, such as splitting, merging, etc.
  • the names of the parameters shown in the headers of the above tables can also use other names that the communication device can understand, and the values or representations of the parameters can also be other values or representations that the communication device can understand.
  • other data structures can also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables, or hash tables, etc.
  • the predefined terms in this disclosure can be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.
  • FIG. 1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure.
  • the communication system 100 may include a satellite, a terminal, a ground station (such as a gateway (GW)), and a third device.
  • the terminal can access the satellite, and the satellite can access a data network through the ground station.
  • the satellite can provide network services to the terminal through the data network.
  • the third device may be a ground device different from the terminal or the ground station, and this third device can be connected to the terminal or the ground station to perform network communication with the terminal or the ground station.
  • the third device may include, for example, an access network device and/or a core network device.
  • the terminal includes, for example, a mobile phone, a wearable device, an Internet of Things (IoT) device, or a device with communication capabilities.
  • the device includes, but is not limited to, at least one of the following: automobile, intelligent vehicle, tablet computer, computer with wireless transceiver capability, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal device in industrial control, wireless terminal device in self-driving, wireless terminal device in remote medical surgery, wireless terminal device in smart grid, wireless terminal device in transportation safety, wireless terminal device in smart city, and wireless terminal device in smart home.
  • VR virtual reality
  • AR augmented reality
  • the access network device is, for example, a node or device that connects a terminal to a wireless network.
  • the access network device may include, but is not limited to, at least one of the following in a 5G communication system: evolved Node B (eNB), next-generation evolved Node B (ng-eNB), next-generation Node B (gNB), node B (NB), home node B (HNB), home evolved node B (HeNB), wireless backhaul device, radio network controller (RNC), base station controller (BSC), base transceiver station (BTS), base band unit (BBU), mobile switching center, base station in a 6G communication system, open RAN, cloud RAN, base station in other communication systems, and access node in a wireless fidelity (WiFi) system.
  • eNB evolved Node B
  • ng-eNB next-generation evolved Node B
  • gNB next-generation Node B
  • gNB next-generation Node B
  • NB node B
  • HNB home node B
  • the technical solutions of this disclosure can be applied to the Open RAN architecture.
  • the interfaces between or within access network devices involved in the embodiments of this disclosure can be transformed into internal interfaces of Open RAN.
  • the processes and information interactions between these internal interfaces can be implemented by software or programs.
  • the access network device may be composed of a central unit (CU) and a distributed unit (DU).
  • the CU may also be called a control unit.
  • the CU-DU structure can separate the protocol layer of the access network device. Some protocol layer functions are centrally controlled by the CU, while the remaining part or all of the protocol layer functions are distributed in the DU and centrally controlled by the CU. However, this is not the only possibility.
  • the core network device may be a single device comprising one or more network elements, or multiple devices or a group of devices, each comprising all or part of one or more network elements.
  • Network elements may be virtual or physical.
  • the core network may include, for example, at least one of an Evolved Packet Core (EPC), a 5G Core Network (5GCN), and a Next Generation Core (NGC).
  • EPC Evolved Packet Core
  • 5GCN 5G Core Network
  • NTC Next Generation Core
  • the core network device may also be a location management function network element.
  • the location management function network element includes a location server, which may be implemented as any of the following: a Location Management Function (LMF), an Enhanced Serving Mobile Location Centre (E-SMLC), a Secure User Plane Location (SUPL), and a Secure User Plane Location Platform (SUPLLP).
  • LMF Location Management Function
  • E-SMLC Enhanced Serving Mobile Location Centre
  • SUPL Secure User Plane Location
  • SUPLLP Secure User Plane Location Platform
  • the following embodiments of this disclosure can be applied to the communication system 100 shown in FIG1, or to some of the main bodies, but are not limited thereto.
  • the main bodies shown in FIG1 are illustrative.
  • the communication system may include all or some of the main bodies in FIG1, or may include other main bodies outside of FIG1.
  • the number and form of each main body are arbitrary.
  • the connection relationship between the main bodies is illustrative.
  • the main bodies may not be connected or may be connected.
  • the connection can be in any way, it can be a direct connection or an indirect connection, it can be a wired connection or a wireless connection.
  • LTE Long Term Evolution
  • LTE-A LTE-Advanced
  • SUPER 3G IMT-Advanced
  • 4th generation mobile communication system 4th generation mobile communication system
  • 5G 5th generation mobile communication system
  • 5G new radio NR
  • Future Radio Access FX
  • RAT New Radio
  • NR New Radio
  • NX New radio access
  • FX Future generation radio access
  • GSM Global System for Mobile communications
  • CDMA2000 Ultra Mobile Broadband
  • UMB Ultra Mobile Broadband
  • IEEE 802.11 Wi-Fi
  • IEEE 802.16 WiMAX
  • IEEE 802.20 Ultra-Wideband
  • Bluetooth a registered trademark
  • D2D Device-to-Device
  • M2M Machine-to-Machine
  • IoT Internet of Things
  • V2X Vehicle-to-Everything
  • V2X Vehicle-to-Everything
  • Figure 2A is an interactive schematic diagram of a handover strategy determination method according to an embodiment of the present disclosure. As shown in Figure 2A, this embodiment of the present disclosure relates to a handover strategy determination method for a communication system 100, the method comprising:
  • Step 2101 The ground station determines the relevant parameters of at least one first satellite.
  • satellites typically need to connect to a data network via a ground station in order to provide network services to the terminals connected to the satellite.
  • the first satellite may be a visible satellite of a ground station.
  • visible satellite here may be understood as, for example, a satellite capable of communication, connection, or service provision.
  • the relevant parameters of the first satellite may include at least one of the following:
  • the signal strength between the ground station and the first satellite which can be, for example, the signal-to-noise ratio (SNR).
  • SNR signal-to-noise ratio
  • the first satellite may include at least one of the following: a low Earth orbit (LEO) satellite, a medium Earth orbit (MEO) satellite, or a highly elliptical orbit (HEO) satellite.
  • LEO low Earth orbit
  • MEO medium Earth orbit
  • HEO highly elliptical orbit
  • Step 2102 The third satellite sends at least one set of parameters related to the second satellite to the ground station.
  • the third satellite may be the satellite that the ground station is currently communicating with, or referred to as: the satellite that the ground station is currently connected to.
  • the second satellite may be a visible satellite of the third satellite, and the relevant parameters of the second satellite may include at least one of the following:
  • the signal strength between the third satellite and the second satellite such as the signal strength (SNR).
  • the second satellite may include at least one of the following: a LEO satellite, a MEO satellite, or a HEO satellite.
  • the ground station may first request relevant parameters of at least one second satellite from the third satellite, and then the third satellite may send the relevant parameters of at least one second satellite to the ground station.
  • steps 2101 and 2102. can be executed simultaneously, or steps 2101 can be executed before or after steps 2102.
  • Step 2103 The ground station inputs the relevant parameters of at least one first satellite and/or the relevant parameters of at least one second satellite into the first model to obtain the switching strategy output by the first model.
  • the first model is deployed on a ground station and can be used for reinforcement learning or deep reinforcement learning.
  • the switching strategy can be used by the ground station to switch satellites.
  • the switching strategy output by the first model can be the switching strategy that achieves the first effect among the alternative switching strategies.
  • the aforementioned "at least one relevant parameter of the first satellite and/or at least one relevant parameter of the second satellite" can be collectively referred to as the network environment parameters of the NTN where the ground station is located. These parameters can reflect the network environment of the NTN where the ground station is located, and the first model can learn (e.g., through reinforcement learning or deep reinforcement learning) a switching strategy with better performance from the alternative switching strategies based on the input network environment parameters and output it.
  • the alternative switching strategy may include at least one of the following:
  • the first strategy is: the nearest satellite handover strategy based on Random Access Channel-less (RACH-less);
  • the second strategy is: a nearest satellite handover strategy that is not based on RACH-less.
  • the third strategy is a switching strategy based on RACH-less and satellite visibility.
  • the fourth strategy is a handover strategy based on satellite visibility but not on RACH-less.
  • the fifth strategy is a handover strategy based on RACH-less and carrier-to-interference-and-noise ratio (CINR).
  • CINR carrier-to-interference-and-noise ratio
  • the sixth strategy is a switching strategy based on CINR but not on RACH-less.
  • the aforementioned “RACH-less” can refer to omitting the random access procedure when switching from a source satellite to a target satellite, thereby reducing... Reduce the time consumed by the random access procedure during handover and shorten the handover interruption latency.
  • the above-mentioned “not RACH-less” can mean that the random access procedure is not omitted when handover from the source satellite to the target satellite.
  • “not RACH-less” may also be called “legacy handover” or other names. This disclosure does not specifically limit it.
  • Figure 2B is a flowchart illustrating the nearest satellite handover strategy according to an embodiment of this disclosure.
  • the visibility range of the ground station includes satellite #1, satellite #2, and satellite #3.
  • satellite #2 is the closest satellite to the ground station, so the ground station can connect to satellite #2.
  • the nearest satellite to the ground station changes from satellite #2 to satellite #3.
  • the ground station can switch from satellite #2 to satellite #3, so that the ground station always connects to the nearest satellite.
  • the aforementioned "switching strategy based on satellite visibility” can refer to: as the satellite moves, when the satellite currently connected to the ground station is about to move outside the ground station's visibility range, the ground station switches from the currently connected satellite to the ground station's most visible satellite.
  • the "most visible satellite” can be understood, for example, as the most distant visible satellite from the ground station, and in the following moments, as the satellite moves, the distance between this satellite and the ground station gradually decreases until it becomes the closest satellite to the ground station, after which the distance between this satellite and the ground station gradually increases, and it gradually moves outside the ground station's visibility range.
  • Figure 2C is a flowchart illustrating the switching strategy based on satellite visibility according to an embodiment of this disclosure.
  • satellite #1 As shown in Figure 2C, at time t(i), the satellite connected to the ground station is satellite #1. Due to the mobility of the satellite, at time t(i+1), satellite #1 is about to move outside the ground station's visibility range. At this time, satellite #4 is the ground station's most visible satellite, so the ground station can switch from satellite #1 to satellite #4.
  • the aforementioned "CINR-based handover strategy" can refer to the following: when the CINR between the ground station and the currently connected satellite meets a first condition, the ground station performs satellite handover, switching to a visible satellite with a CINR higher than a preset threshold or switching to the satellite with the best CINR.
  • the first condition can be, for example, a CINR that is 3 dB lower than the maximum reference level.
  • the first effect described above may include at least one of the following:
  • the link spectral efficiency is greater than the first value, for example: the spectral efficiency of the link after handover is greater than the first value;
  • the switching rate is less than the second value; alternatively, the switching rate may refer to the "number of switching".
  • the handover success rate is greater than the third value
  • the switching delay is less than the fourth value
  • the switching interruption time is less than the fifth value
  • the Doppler frequency shift is less than the sixth value.
  • the switching strategy output by the first model can be the best alternative switching strategy among the alternative switching strategies that achieve the first effect; or, in other embodiments, the switching strategy output by the first model can be any alternative switching strategy among the alternative switching strategies that achieve the first effect.
  • the first model when the first model outputs the switching strategy based on the network environment parameters of the ground station, it outputs the "alternative switching strategy that can achieve the first effect when switching in the current network environment". This ensures that the switching strategy output by the first model is not only adapted to the current network environment, but also achieves a better switching effect, thereby improving the switching performance when switching satellites.
  • the model algorithm of the first model described above may include at least one of the following algorithms: Q-Learning algorithm, Deep Q-Network (DQN) algorithm, Double Deep Q-Network (DDQN) algorithm, and Actor-Critic algorithm.
  • Q-Learning algorithm Deep Q-Network (DQN) algorithm
  • DQN Deep Q-Network
  • DDQN Double Deep Q-Network
  • Actor-Critic algorithm Actor-Critic algorithm.
  • the aforementioned "Q-Learning algorithm” is a reinforcement learning algorithm
  • the aforementioned "DQN algorithm, DDQN algorithm, and Actor-Critic algorithm” are deep reinforcement learning algorithms.
  • reinforcement learning can be understood as a method of learning optimal behavioral strategies through the interaction between an agent and its environment. The agent observes the current state in the environment, takes action and receives rewards or punishments, and then adjusts its strategy based on this feedback. The goal of reinforcement learning is to obtain the maximum cumulative reward through interaction with the environment. Reinforcement learning does not require labeled training data but learns through trial and error.
  • An important component of reinforcement learning is the Markov Decision Process (MDP), which defines the environment model and the agent's decision-making strategy.
  • MDP Markov Decision Process
  • Deep reinforcement learning combines the perceptual capabilities of deep learning with the decision-making capabilities of reinforcement learning. It learns complex features and patterns by training a multi-layered neural network (e.g., the first model in this embodiment), so that the neural network can directly derive the optimal behavioral strategy based on the input image and/or data. Furthermore, deep reinforcement learning requires training data to train the neural network.
  • a multi-layered neural network e.g., the first model in this embodiment
  • Q-Learning algorithm i.e., reinforcement learning algorithm
  • DQN algorithm i.e., deep reinforcement learning algorithm
  • the first model when the modeling method of the first model is the "Q-Learning algorithm (i.e., reinforcement learning algorithm)," the first model needs to calculate the Q value corresponding to each alternative switching strategy based on the input network environment parameters and using the Q-value calculation formula. Different alternative switching strategies have different Q values. A larger Q value indicates a better switching effect (i.e., the aforementioned first effect) for the alternative switching strategy. The first model can then output the alternative switching strategy with the better switching effect based on the Q value.
  • Figure 2D shows the Q-value calculation formula according to an embodiment of this disclosure. As shown in Figure 2D, it includes three formulas. The first formula can be used to calculate the Q value, and the second and third formulas can be used to represent the relationship between Q values at adjacent time points.
  • Q(s,a) represents the Q value corresponding to the alternative switching strategy
  • r(s,a) and R(s,a) represent the rewards for performing action a corresponding to the alternative switching strategy in the current state s.
  • r(s,a) and R(s,a) can be determined by referring to the input network environment parameters.
  • the switching effect i.e., the aforementioned first effect
  • r(s,a) and R(s,a) are higher;
  • Q ⁇ sub>t-1 ⁇ /sub> r(s,a) and Q ⁇ sub>t-1 ⁇ /sub> (s,a) both represent the Q value at the current time, such as: the Q value in the state before switching;
  • Q ⁇ sub>t ⁇ /sub> (s,a) represents the Q value at the next time, such as: the Q value in the state after satellite switching based on the alternative switching strategy;
  • ⁇ represents the learning rate factor, 0 ⁇ 1;
  • ⁇ in Figure 2D is the discount factor, max a'Q (s',a') and max a Q(s',a) represents the maximum future reward under the new state and new action.
  • the "new state” here can be understood as the state after satellite switching based on alternative switching strategies, and the "new action” here can be understood as all possible actions under the new state.
  • the first model can learn and output a better switching strategy by performing MDP based on the formula shown in Figure 2D.
  • the first model when the modeling method of the first model is the "DQN algorithm (i.e., deep reinforcement learning algorithm)," the first model does not need to use formula calculations. Instead, it can directly determine the Q-values corresponding to each alternative handover strategy based on the input network environment parameters, and output the alternative handover strategy with better handover performance based on the Q-values. Therefore, the first model needs to learn the calculation method of the Q-values so that it can directly determine the Q-values corresponding to the alternative handover strategies based on the network environment parameters. This requires training the first model to learn the Q-value calculation method. In some embodiments, a sample set can be determined first.
  • This sample set may include the network environment parameters of the NTN at a historical time of the ground station, and the Q-value corresponding to a certain handover strategy at that historical time. Then, the network environment parameters at that historical time are input into the first model, and the Q-value output by the first model is determined. The model parameters of the first model are then adjusted based on the loss function until the loss function converges.
  • Figure 2E shows the calculation formula of the loss function according to an embodiment of this disclosure. As shown in Figure 2E, TD in the loss function has the same meaning as TDt in Figure 2D. r in the loss function has the same meaning as R(s,a) and r(s,a) in Figure 2D.
  • max a' Q(s',a', ⁇ ') in the loss function has the same meaning as max a' Q(s',a') and max a Q(s',a) in Figure 2D.
  • Q(s,a, ⁇ ) in the loss function has the same meaning as Q t-1 (s,a) and Q t-1 r(s,a) in Figure 2D.
  • ⁇ is a random parameter.
  • Step 2104 The ground station performs the first operation to switch satellites.
  • the first operation can be: performing satellite handover directly based on the handover strategy output by the first model. That is, the ground station itself has decision-making power and can decide on the specific handover strategy. For example, when the satellite handover of the ground station is triggered by a Conditional Handover (CHO), the ground station can directly perform satellite handover based on the handover strategy output by the first model.
  • CHO Conditional Handover
  • the first operation may include: reporting the handover strategy output by the first model to a third satellite, receiving the handover strategy sent by the third satellite, and then performing a satellite handover based on the handover strategy sent by the third satellite; wherein the handover strategy sent by the third satellite may be the same as or different from the handover strategy output by the first model. That is, the ground station does not have decision-making power and needs to perform the handover based on the instructions of the third satellite. For example, when the ground station autonomously determines that a satellite handover needs to be performed, it can report the handover strategy output by the first model to the third satellite so that the subsequent satellite handover can be performed based on the handover strategy sent by the third satellite.
  • a ground station when a ground station needs to perform satellite handover, it uses the first model to determine the corresponding handover strategy based on the network environment parameters of the NTN where it is located. This ensures that the determined handover strategy matches the ground station's network environment. Therefore, when a ground station performs satellite handover, it adopts an appropriate handover strategy based on the different network environments it is in, improving the flexibility of satellite handover, avoiding waste of handover resources, and meeting the real-time requirements of handover services. Furthermore, since the handover strategy in this embodiment is determined based on the network environment of the ground station and is independent of the "satellite type," the handover strategy of this method is not limited by the satellite type. This allows for handover between different types of satellites, ensuring the collaborative operation of different types of satellites and guaranteeing communication performance.
  • the switching strategy determination method involved in the embodiments of this disclosure may include at least one of steps 2101 to 2104.
  • step 2101 may be implemented as an independent embodiment
  • step 2102 may be implemented as an independent embodiment
  • step 2103 may be implemented as an independent embodiment
  • step 2101+S2102 may be implemented as an independent embodiment, but is not limited thereto.
  • each step can be independent, arbitrarily combined or exchanged in order, optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other implementations or other embodiments.
  • Figure 2F is an interactive schematic diagram of a handover strategy determination method according to an embodiment of the present disclosure. As shown in Figure 2F, this embodiment of the disclosure relates to a handover strategy determination method for a communication system 100, the method comprising:
  • Step 2201 The terminal determines the relevant parameters of at least one first satellite.
  • the first satellite may be a visible satellite of the terminal.
  • visible satellite here may be understood as, for example, a satellite that is capable of communication, connection or service.
  • the relevant parameters of the first satellite may include at least one of the following:
  • the distance between the terminal and the first satellite is the distance between the terminal and the first satellite
  • the signal strength between the terminal and the first satellite which can be, for example, the signal strength (SNR).
  • SNR signal strength
  • the first satellite may include at least one of the following: a LEO satellite, a MEO satellite, or a HEO satellite.
  • Step 2202 The third satellite sends relevant parameters of at least one second satellite to the terminal.
  • the third satellite may be the satellite the terminal is currently communicating with, or the satellite the terminal is currently connected to.
  • the second satellite may be a visible satellite of the third satellite; the relevant parameters of the second satellite can be referred to the description in the above embodiments.
  • the terminal may first request relevant parameters of at least one second satellite from the third satellite, and then the third satellite may send the relevant parameters of at least one second satellite to the terminal.
  • steps 2201 and 2202. can be executed simultaneously, or steps 2201 can be executed before or after steps 2202.
  • Step 2203 The terminal inputs the relevant parameters of at least one first satellite and/or the relevant parameters of at least one second satellite into the first model to obtain the switching strategy output by the first model.
  • steps 2202-2203 please refer to the embodiment described in Figure 2A above.
  • Step 2204 The terminal performs the first operation to perform satellite handover.
  • the first operation can be: performing satellite handover directly based on the handover strategy output by the first model. That is, the terminal itself has decision-making power and can decide on the specific handover strategy. For example, when the terminal's satellite handover is triggered by CHO, the terminal can directly perform satellite handover based on the handover strategy output by the first model.
  • the first operation may include: reporting the handover strategy output by the first model to a third satellite, receiving the handover strategy sent by the third satellite, and then performing a satellite handover based on the handover strategy sent by the third satellite; wherein the handover strategy sent by the third satellite may be the same as or different from the handover strategy output by the first model. That is, the terminal does not have decision-making power and needs to perform the handover based on the instructions of the third satellite. For example, when the terminal autonomously determines that a satellite handover is to be performed, it can report the handover strategy output by the first model to the third satellite so that the subsequent satellite handover can be performed based on the handover strategy sent by the third satellite.
  • a terminal when a terminal needs to perform satellite handover, it utilizes the first model to determine the corresponding handover strategy based on the network environment parameters of the NTN where the terminal is located. This ensures that the determined handover strategy matches the network environment of the ground station. Therefore, when the terminal performs satellite handover, it adopts an appropriate handover strategy based on the different network environments in which the terminal is located, improving the flexibility of satellite handover, avoiding waste of handover resources, and meeting the real-time requirements of handover services. Furthermore, since the handover strategy in this embodiment is determined based on the network environment of the terminal and is independent of the "satellite type," the handover strategy of this method is not limited by the satellite type. This allows for handover between different types of satellites, ensuring the collaborative operation of different types of satellites and guaranteeing communication performance.
  • the switching strategy determination method involved in the embodiments of this disclosure may include at least one of steps 2201 to 2204.
  • step 2201 may be implemented as a standalone embodiment
  • step 2202 may be implemented as a standalone embodiment
  • step 2203 may be implemented as a standalone embodiment
  • step 2201+S2202 may be implemented as a standalone embodiment, but is not limited thereto.
  • each step can be independent, arbitrarily combined or exchanged in order, optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other implementations or other embodiments.
  • Figure 2G is an interactive schematic diagram of a handover strategy determination method according to an embodiment of the present disclosure. As shown in Figure 2G, the embodiments of the present disclosure involve... and a method for determining a handover strategy, used in communication system 100, the method comprising:
  • Step 2301 The second device sends at least one relevant parameter of the first satellite to the satellite on which the first model is deployed.
  • the second device can be a device that requires satellite switching, such as a terminal or ground station.
  • the first satellite can be a visible satellite of the second device.
  • the relevant parameters of the first satellite may include at least one of the following:
  • the distance between the second device and the first satellite is the distance between the second device and the first satellite
  • the signal strength between the second device and the first satellite such as the signal strength (SNR).
  • the "satellite that has deployed the first model" mentioned above can be any satellite, for example, it can be the first satellite or not.
  • the second device can know in advance which satellites have deployed the first model. When the second device needs to switch satellites, it can send relevant parameters of at least one of the first satellites to the satellites that have deployed the first model.
  • Step 2302 Determine the relevant parameters of at least one second satellite using the satellites that have deployed the first model.
  • the second satellite can be a visible satellite of a third satellite, which can be the satellite the second device is currently communicating with, or referred to as the satellite the second device is currently connected to.
  • the relevant parameters of the second satellite can be referred to the description in the above embodiments.
  • the aforementioned "satellite deploying the first model” may also be a second or a third satellite, or it may not be a second or a third satellite.
  • the "satellite deploying the first model" when the "satellite deploying the first model" is a third satellite, it can autonomously determine the relevant parameters of at least one second satellite based on the satellites it can see; when the "satellite deploying the first model" is not a third satellite, it can first determine the second satellite visible to the third satellite based on the satellite's ephemeris information, and then determine the relevant parameters of at least one second satellite.
  • steps 2301 and 2302 can be executed simultaneously, or steps 2301 can be executed before or after steps 2302.
  • Step 2303 The satellites that have deployed the first model input at least one relevant parameter of the first satellite and/or at least one relevant parameter of the second satellite into the first model to obtain the switching strategy output by the first model.
  • step 2303 please refer to the embodiment described in Figure 2A above.
  • Step 2304 The satellite that has deployed the first model instructs the second device on the switching strategy output by the first model.
  • Step 2305 The second device performs satellite handover based on the handover strategy output by the first model.
  • the satellites deployed with the first model will use the first model to determine the corresponding handover strategy based on the network environment parameters of the NTN where the second device is located. This ensures that the determined handover strategy matches the network environment of the second device. Therefore, when the second device performs satellite handover, it will adopt an appropriate handover strategy based on the different network environments in which it is located, improving the flexibility of satellite handover, avoiding waste of handover resources, and meeting the real-time requirements of handover services. Furthermore, since the handover strategy in this embodiment is determined based on the network environment of the second device and is independent of the "satellite type," the handover strategy of this method is not limited by the satellite type. This allows for handover between different types of satellites, ensuring the collaborative work of different types of satellites and guaranteeing communication performance.
  • the switching strategy determination method involved in the embodiments of this disclosure may include at least one of steps 2301 to 2305.
  • step 2301 may be implemented as a standalone embodiment
  • step 2302 may be implemented as a standalone embodiment
  • step 2303 may be implemented as a standalone embodiment
  • step 2301+S2302 may be implemented as a standalone embodiment, but is not limited thereto.
  • each step can be independent, arbitrarily combined or exchanged in order, optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other implementations or other embodiments.
  • Figure 2H is an interactive schematic diagram of a handover strategy determination method according to an embodiment of the present disclosure. As shown in Figure 2H, this embodiment of the present disclosure relates to a handover strategy determination method for a communication system 100, the method comprising:
  • Step 2401 The second device sends at least one parameter of the first satellite to the third device that has deployed the first model.
  • the second device can be a device that requires satellite switching, such as a terminal or ground station.
  • the first satellite can be a visible satellite of the second device. Detailed descriptions of the relevant parameters of the first satellite can be found in the embodiments described above.
  • the "third device that has deployed the first model" mentioned above can be a ground device that is different from the second device, for example, it can be a network device, such as an access network device and/or a core network device.
  • the second device can know in advance which third device has deployed the first model.
  • the second device needs to perform satellite switching, it can send relevant parameters of at least one first satellite to the third device that has deployed the first model.
  • Step 2402 The third satellite transmits relevant parameters of at least one second satellite to the third device that deployed the first model.
  • the third satellite may be the satellite that the second device is currently communicating with, or the satellite that the second device is currently connected to.
  • the second satellite may be a visible satellite of the third satellite.
  • the relevant parameters of the second satellite can be referred to the embodiments described above.
  • the third satellite can know in advance which device has deployed the first model.
  • the third satellite determines that the second device needs to perform satellite switching, it can send relevant parameters of at least one second satellite to the third device that has deployed the first model.
  • steps 2401 and 2402 can be executed simultaneously, or steps 2401 can be executed before or after steps 2402.
  • Step 2403 The third device inputs relevant parameters of at least one first satellite and/or relevant parameters of at least one second satellite into the first model to obtain the switching strategy output by the first model.
  • step 2403 please refer to the embodiment described in Figure 2A above.
  • Step 2404 The third device performs the second operation to enable the second device to perform satellite switching.
  • the second operation may be, for example, the third device directly instructing the second device on the switching strategy output by the first model, so that the second device performs satellite switching based on the switching strategy.
  • the second operation may be as follows: the third device reports the handover strategy output by the first model to the third satellite currently communicating with the second device, so that the third satellite determines the final handover strategy.
  • the third satellite can then indicate the final handover strategy to the second device, thereby allowing the second device to perform satellite handover based on the handover strategy.
  • the final handover strategy determined by the third satellite may be the same as or different from the handover strategy output by the first model.
  • the third device when the second device needs to perform satellite handover, the third device, which has deployed the first model, will use the first model to determine the corresponding handover strategy based on the network environment parameters of the NTN where the second device is located. This ensures that the determined handover strategy is compatible with the network environment of the second device. Therefore, when the second device performs satellite handover, it will adopt a suitable handover strategy based on the different network environments in which it is located, improving the flexibility of satellite handover, avoiding waste of handover resources, and meeting the real-time requirements of handover services. Furthermore, since the handover strategy in this embodiment is determined based on the network environment of the second device and is independent of the "satellite type," the handover strategy of this method is not limited by the satellite type. This allows for handover between different types of satellites, ensuring the collaborative operation of different types of satellites and guaranteeing communication performance.
  • the switching strategy determination method involved in the embodiments of this disclosure may include at least one of steps 2401 to 2404.
  • step 2401 may be implemented as a standalone embodiment
  • step 2402 may be implemented as a standalone embodiment
  • step 2403 may be implemented as a standalone embodiment
  • step 2401+S2402 may be implemented as a standalone embodiment, but is not limited thereto.
  • each step can be independent, arbitrarily combined or exchanged in order, optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other implementations or other embodiments.
  • Figure 3 is an interactive schematic diagram of a handover strategy determination method according to an embodiment of the present disclosure. As shown in Figure 3, this embodiment of the present disclosure relates to a handover strategy determination method for a first device, the method comprising:
  • Step 3101 Determine the first parameter.
  • Step 3102 Input the first parameter into the first model to obtain the switching strategy output by the first model.
  • the first parameter is used to indicate the network environment parameters of the non-terrestrial network NTN where the second device is located; the second device is a device that needs to perform satellite handover.
  • the first model is used for reinforcement learning or deep reinforcement learning
  • the switching strategy is used by the second device to perform satellite switching.
  • the switching strategy output by the first model is the switching strategy that achieves the first effect among the alternative switching strategies.
  • the alternative switching strategy includes at least one of the following:
  • the first strategy is: the nearest satellite handover strategy based on RACH-less random access channel;
  • the second strategy is: a nearest satellite handover strategy not based on RACH-less;
  • the third strategy is a switching strategy based on RACH-less and satellite visibility.
  • the fourth strategy is a handover strategy based on satellite visibility but not on RACH-less.
  • the fifth strategy is a handover strategy based on RACH-less and carrier interference-to-noise ratio (CINR).
  • CINR carrier interference-to-noise ratio
  • the sixth strategy is a switching strategy based on CINR but not on RACH-less.
  • the first effect includes at least one of the following:
  • the link spectral efficiency is greater than the first value
  • the switching rate is less than the second value
  • the handover success rate is greater than the third value
  • the switching delay is less than the fourth value
  • the switching interruption time is less than the fifth value
  • the Doppler frequency shift is less than the sixth value.
  • the switching strategy output by the first model is: the best-performing alternative switching strategy among the alternative switching strategies that achieve the first effect; or
  • the switching strategy output by the first model is any one of the alternative switching strategies that achieves the first effect.
  • the network environment parameters of the NTN where the second device is located include at least one of the following:
  • At least one relevant parameter of a first satellite is a visible satellite of the second device;
  • At least one second satellite's relevant parameters the second satellite is a visible satellite of the third satellite that the second device is currently communicating with.
  • the relevant parameters of the first satellite include at least one of the following:
  • the distance between the second device and the first satellite is the distance between the second device and the first satellite
  • the signal strength between the second device and the first satellite is the signal strength between the second device and the first satellite.
  • the relevant parameters of the second satellite include at least one of the following:
  • the signal strength between the third satellite and the second satellite is the signal strength between the third satellite and the second satellite.
  • the first satellite and the second satellite each include at least one of the following: a low Earth orbit (LEO) satellite, a medium Earth orbit (MEO) satellite, and a highly elliptical orbit (HEO) satellite.
  • LEO low Earth orbit
  • MEO medium Earth orbit
  • HEO highly elliptical orbit
  • the first device includes at least one of any satellite, a second device, and a third device;
  • the second device includes a terminal or ground station; the third device is a ground device different from the second device.
  • the first device is any satellite, and determining the first parameter includes:
  • the first device determines the relevant parameters of at least one second satellite based on the implementation.
  • the first device is the second device, and determining the first parameter includes:
  • the first device determines relevant parameters of at least one first satellite based on the implementation
  • the first device is a third device, and determining the first parameter includes:
  • the first device is any satellite, and the method further includes:
  • the second device is instructed to use the handover strategy output by the first model, so that the second device can perform satellite handover based on the handover strategy.
  • the first device is a second device, and the method further includes:
  • Satellite handover is performed based on the handover strategy output by the first model
  • the device reports the switching strategy output by the first model to the third satellite currently communicating with the second device, and receives the switching strategy sent by the third satellite, and performs satellite switching based on the switching strategy sent by the third satellite; wherein the switching strategy sent by the third satellite is the same as or different from the switching strategy output by the first model.
  • the first device is a third device, and the method further includes:
  • the handover strategy output by the first model is reported to the third satellite currently communicating with the second device, so that the third satellite can determine the final handover strategy; wherein the final handover strategy determined by the third satellite is the same as or different from the handover strategy output by the first model.
  • the model algorithm of the first model includes at least one of the following algorithms: Q-Learning algorithm, Deep Q-Network (DQN) algorithm, Dual Deep Q-Network (DDQN) algorithm, and Actor-Evaluation (Critic) algorithm.
  • Q-Learning algorithm Deep Q-Network (DQN) algorithm
  • DQN Deep Q-Network
  • DDQN Dual Deep Q-Network
  • Critic Actor-Evaluation
  • steps 3101-3102 For a detailed description of steps 3101-3102, please refer to the above embodiment description.
  • the determination method involved in the embodiments of this disclosure may include at least one of steps S3101 to S3102.
  • step S3101 may be implemented as an independent embodiment
  • step S3102 may be implemented as an independent embodiment
  • steps S3101 to S3102 may be implemented as independent embodiments, but are not limited thereto.
  • each step can be independent, arbitrarily combined or exchanged in order, optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other implementations or other embodiments.
  • FIG. 4A is an interactive schematic diagram illustrating the handover strategy determination method according to an embodiment of this disclosure.
  • FIG. 4B is a schematic diagram of the structure of the deep learning method according to an embodiment of the present disclosure.
  • a Markov decision process is a set of five data components (S, A, R, P, ⁇ ).
  • S is a finite set of states
  • A is a finite set of actions
  • A(s) is a finite set of activities that can be obtained from the states
  • P is the probability that being in state s at time t results in being in state s' at time t+1;
  • R is the direct reward obtained after switching from state s' by performing action a;
  • ⁇ is the discount factor, which represents the key difference between future rewards and current rewards.
  • the agent knows which action to choose to obtain the greatest reward.
  • FIG. 4D is a schematic diagram of the Q-learning process according to an embodiment of this disclosure. Therefore, the Q-learning process can be determined according to the formula shown in Figure 2D above and Figure 4D. Where: TDt is the instantaneous value to be calculated. ⁇ is the learning rate factor.
  • FIG. 4E is a schematic diagram of the neural network of a deep Q-network according to an embodiment of this disclosure.
  • the loss function is determined according to the formula shown in Figure 2E above.
  • ⁇ is a random parameter
  • a Q-table function approximator is used. Because the DQN agent has its own action space, a multi-output Q-table function can be used.
  • a Q-table vector takes a single observation as input and returns a single vector with as many elements and feasible actions as outputs as possible.
  • the value of each output element represents the discounted cumulative long-term reward.
  • the reward function in order to simultaneously guarantee the performance of various services in different aspects, is based on link spectral efficiency, HO rate, and HO... Configure settings for success rate, HO delay, HO interruption time, and Doppler shift.
  • DQN performs the following steps:
  • a state set introduces a state into the network; the output is Q tables of a single action.
  • the agent selects and executes the policy.
  • the state set returns state s', the regular part r is the result of action a, and [s,a,r,s'] is stored in memory.
  • the experimental samples were divided into several batches for neural network training.
  • the process iterates through the last N cycles.
  • the HO type nearest satellite HO, maximum visibility HO, CINR-based HO
  • the HO type is allowed to make a decision and receive feedback on state changes multiple times to learn how to make the right decision on its own.
  • the neural network is trained on an experience basis as an agent, and the number of iterations per cycle and the maximum number of training sets can be adjusted according to the training effect.
  • the algorithm trains the model by ranking discrete actions based on the output.
  • the state of the environment can be obtained based on other factors considered in the reward function, and finally, by ranking the actions, an optimal HO (House of Interest) type action is selected.
  • HO House of Interest
  • This disclosure also provides an apparatus for implementing any of the above methods.
  • an apparatus is provided that includes units or modules for implementing the steps performed by the terminal in any of the above methods.
  • another apparatus is provided that includes units or modules for implementing the steps performed by a network device (e.g., an access network device, a core network functional node, a core network device, etc.) in any of the above methods.
  • a network device e.g., an access network device, a core network functional node, a core network device, etc.
  • the division of units or modules in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated.
  • the units or modules in the device can be implemented by a processor calling software: for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the units or modules in the above device.
  • the processor can be, for example, a general-purpose processor, such as a Central Processing Unit (CPU) or a microprocessor, and the memory can be internal or external to the device.
  • the units or modules in the device can be implemented in the form of hardware circuits.
  • the functionality of some or all of the units or modules can be achieved through the design of these hardware circuits, which can be understood as one or more processors.
  • the hardware circuit is an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • the functionality of some or all of the units or modules is achieved through the design of the logical relationships between the components within the circuit.
  • the hardware circuit can be implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby achieving the functionality of some or all of the units or modules. All units or modules of the above device can be implemented entirely through processor-called software, entirely through hardware circuits, or partially through processor-called software with the remaining parts implemented through hardware circuits.
  • PLD programmable logic device
  • the processor is a circuit with signal processing capabilities.
  • the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP).
  • the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable.
  • the processor is a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA.
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules.
  • it can also be hardware circuits designed for artificial intelligence, which can be understood as ASICs, such as Neural Network Processing Unit (NPU), Tensor Processing Unit (TPU), Deep Learning Processing Unit (DPU), etc.
  • ASICs such as Neural Network Processing Unit (NPU), Tensor Processing Unit (TPU), Deep Learning Processing Unit (DPU), etc.
  • Figure 5 is a schematic diagram of the structure of the first device proposed in an embodiment of this disclosure. As shown in Figure 5, it includes:
  • the processing module is used to determine a first parameter; the first parameter is used to indicate the network environment parameters of the non-terrestrial network NTN where the second device is located; the second device is a device that needs to perform satellite handover.
  • the processing module is further configured to input the first parameter into the first model to obtain the switching strategy output by the first model; wherein the first model is used to perform reinforcement learning or deep reinforcement learning, and the switching strategy is used by the second device to perform satellite switching.
  • the processing module is used to execute the steps related to "processing" performed by the first device in any of the above methods.
  • the first device further includes a transceiver module, which is used to execute the steps related to "sending and receiving" performed by the first device in any of the above methods.
  • FIG. 6A is a schematic diagram of the structure of the communication device 6100 proposed in an embodiment of this disclosure.
  • the communication device 6100 can be a network device (e.g., access network device, core network device, etc.), a terminal (e.g., user equipment or the first device described above), a chip, chip system, or processor that supports the network device in implementing any of the above methods, or a chip, chip system, or processor that supports the terminal in implementing any of the above methods.
  • the communication device 6100 can be used to implement the methods described in the above method embodiments; for details, please refer to the descriptions in the above method embodiments.
  • the communication device 6100 includes one or more processors 6101.
  • the processor 6101 can be a general-purpose processor or a dedicated processor, such as a baseband processor or a central processing unit (CPU).
  • the baseband processor can be used to process communication protocols and communication data, while the CPU can be used to control communication devices (e.g., base stations, baseband chips, terminal devices, terminal device chips, DUs or CUs, etc.), execute programs, and process program data.
  • the processor 6101 is used to invoke instructions to cause the communication device 6100 to execute any of the above methods.
  • the communication device 6100 further includes one or more memories 6102 for storing instructions.
  • the memories 6102 may also be located outside the communication device 6100.
  • the communication device 6100 further includes one or more transceivers 6103.
  • the communication steps such as sending and receiving in the above method are performed by the transceivers 6103, and other steps are performed by the processor 6101.
  • a transceiver may include a receiver and a transmitter, which may be separate or integrated.
  • transceiver, transceiver unit, transceiver, transceiver circuit, etc. may be used interchangeably; the terms transmitter, transmitting unit, transmitter, transmitting circuit, etc., may be used interchangeably; and the terms receiver, receiving unit, receiver, receiving circuit, etc., may be used interchangeably.
  • the communication device 6100 further includes one or more interface circuits 6104 connected to the memory 6102.
  • the interface circuits 6104 can be used to receive signals from the memory 6102 or other devices, and can be used to send signals to the memory 6102 or other devices.
  • the interface circuits 6104 can read instructions stored in the memory 6102 and send the instructions to the processor 6101.
  • the communication device 6100 described in the above embodiments may be a network device or a terminal, but the scope of the communication device 6100 described in this disclosure is not limited thereto, and the structure of the communication device 6100 may not be limited by FIG. 6a.
  • the communication device may be a standalone device or a part of a larger device.
  • the communication device may be: (1) a standalone integrated circuit IC, or chip, or chip system or subsystem; (2) a collection of one or more ICs, optionally, the IC collection may also include storage components for storing data and programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, terminal device, smart terminal device, cellular phone, wireless device, handheld device, mobile unit, vehicle device, network device, cloud device, artificial intelligence device, etc.; (6) others, etc.
  • Figure 6B is a schematic diagram of the structure of chip 6200 according to an embodiment of this disclosure.
  • the communication device 6100 can be a chip or a chip system, please refer to the schematic diagram of chip 6200 shown in Figure 6B, but it is not limited thereto.
  • Chip 6200 includes one or more processors 6201, which are used to invoke instructions to cause chip 6200 to perform any of the above methods.
  • chip 6200 further includes one or more interface circuits 6202 connected to memory 6203.
  • Interface circuits 6202 can be used to receive signals from memory 6203 or other devices, and can also be used to send signals to memory 6203 or other devices.
  • interface circuit 6202 can read instructions stored in memory 6203 and send those instructions to processor 6201.
  • terms such as interface circuit, interface, transceiver pin, and transceiver can be used interchangeably.
  • chip 6200 further includes one or more memories 6203 for storing instructions.
  • all or part of the memories 6203 may be located outside of chip 6200.
  • the storage medium is an electronic storage medium.
  • the storage medium is a computer-readable storage medium, but not limited thereto; it may also be a storage medium readable by other devices.
  • the storage medium may be a non-transitory storage medium, but not limited thereto; it may also be a temporary storage medium.
  • This disclosure also provides a program product that, when executed by the communication device 6100, causes the communication device 6100 to perform any of the above methods.
  • the program product is a computer program product.
  • This disclosure also proposes a computer program that, when run on a computer, causes the computer to perform any of the above methods.
  • implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof.
  • the computer program can be implemented, in whole or in part, as a computer program product.
  • the computer program product includes one or more computer programs.
  • the computer program When the computer program is loaded and executed on a computer, it generates, in whole or in part, the processes or functions described in the embodiments of this disclosure.
  • the computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.
  • the computer program can be transferred from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media.
  • the available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本公开提出一种切换策略确定方法、通信设备、通信系统、存储介质，方法包括：确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。本公开的方法提高了卫星切换的灵活性，避免了切换资源的浪费，满足了切换业务的实时要求。并且，本公开可以实现不同类型的卫星之间的切换，从而保证了不同类型卫星的协同工作，确保了通信性能。

Description

切换策略确定方法、通信设备、通信系统、存储介质 技术领域
本公开涉及通信技术领域，尤其涉及切换策略确定方法、通信设备、通信系统、存储介质。
背景技术
在非地面网络(Non-terrestrial Network，NTN)系统中，终端或地面站会连接至卫星。可选地，由于卫星是连续运动的，因此卫星对于终端或地面站而言，并非一直可见，由此终端或地面站通常需要进行卫星切换(handover，HO)，例如当终端或地面站所连接的卫星移动至不可见位置时，终端或地面站可以从当前连接卫星切换至其他可见卫星，以此确保卫星服务的连续性。可选地，在一些实施例之中，针对不同类型的卫星会分别固定设置不同的切换策略。
其中，由于不同类型的卫星的切换策略是固定不变，则会导致卫星切换无法根据网络环境自适应变化，使得卫星切换的灵活性较差、且无法满足切换业务的实时要求。并且，当不同类型的卫星对应的切换策略不同时，也无法在不同类型的卫星之间进行切换，从而无法实现不同类型卫星的协同工作，影响通信性能。
发明内容
本公开提出切换策略确定方法、通信设备、通信系统、存储介质。
根据本公开实施例的第一方面，提出了一种切换策略确定方法，由第一设备执行，所述方法包括：
确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
根据本公开实施例的第二方面，提出了一种第一设备，包括：
处理模块，用于确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
所述处理模块，还用于将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
根据本公开实施例的第三方面，提出了一种通信设备，包括：
一个或多处理器；
其中，所述处理器用于调用指令以使得所述通信设备执行第一方面所述的方法。
根据本公开实施例的第四方面，提出了一种通信系统，其特征在于，包括卫星、第二设备、第三设备中的至少之一，其中，所述卫星或者第二设备或者第三设备被配置为实现第一方面所述的方法。
根据本公开实施例的第五方面，提出了一种存储介质，所述存储介质存储有指令，其特征在于，当所述指令在通信设备上运行时，使得所述通信设备执行如第一方面所述的方法。
第六方面，本公开实施例提出了程序产品，上述程序产品被通信设备执行时，使得上述通信设备执行如第一方面所述的方法。
第七方面，本公开实施例提出了计算机程序，当其在计算机上运行时，使得计算机执行如第一方面所述的方法。
第八方面，本公开实施例提出了一种芯片或芯片系统。该芯片或芯片系统包括处理电路，被配置为执行根据上述第一方面所述的方法。
可以理解地，上述终端、网络设备、通信设备、通信系统、存储介质、程序产品、计算机程序均用于执行本公开实施例所提出的方法。因此，其所能达到的有益效果可以参考对应方法中的有益效果，此处不再赘述。
附图说明
本公开上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为本公开实施例提供的一些通信系统的架构示意图；
图2A为本公开再一个实施例所提供的切换策略确定方法的流程示意图；
图2B是根据本公开实施例示出的最近卫星切换策略的流程示意图；
图2C是根据本公开实施例示出的基于卫星可见度的切换策略的流程示意图；
图2D是根据本公开实施例示出的Q值的计算公式；
图2E是根据本公开实施例示出的损失函数的计算公式；
图2F是根据本公开实施例示出的切换策略确定方法的交互示意图；
图2G是根据本公开实施例示出的切换策略确定方法的交互示意图；
图2H是根据本公开实施例示出的切换策略确定方法的交互示意图；
图3是根据本公开实施例示出的切换策略确定方法的交互示意图；
图4A是根据本公开实施例示出的切换策略确定方法的交互示意图；
图4B是根据本公开实施例示出的深度学习方法的结构示意图；
图4C是根据本公开实施例示出的随机奖励的累积函数的计算公式；
图4D是根据本公开实施例示出的Q-learning过程的示意图；
图4E是根据本公开实施例示出的deep Q-network的神经网络示意图；
图5是本公开实施例提出的第一设备的结构示意图；
图6A是本公开一个实施例所提供的一种通信设备的结构示意图；
图6B为本公开一个实施例所提供的一种芯片的结构示意图。
具体实施方式
本公开实施例提出了切换策略确定方法、通信设备、通信系统、存储介质。
第一方面，本公开实施例提出了一种切换策略确定方法，由第一设备执行，所述方法包括：
确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
在上述实施例中，针对需进行卫星切换的第二设备，第一设备会利用第一模型基于第二设备所处的NTN的网络环境参数确定对应的切换策略，以使得所确定出的切换策略是匹配于第二设备的网络环境的，从而当第二设备进行卫星切换时，会基于第二设备所处的不同网络环境来针对性的采用合适的切换策略进行卫星切换，提高了卫星切换的灵活性，避免了切换资源的浪费，满足了切换业务的实时要求。并且，由于本公开实施例中的切换策略是基于第二设备所处的网络环境确定的，与“卫星类型”无关，因此，本公开方法的切换策略不会受限于卫星类型，则可以实现不同类型的卫星之间的切换，从而保证了不同类型卫星的协同工作，确保了通信性能。
结合第一方面的一些实施例，在一些实施例中，所述第一模型输出的切换策略为备选切换策略中达到第一效果的切换策略。
结合第一方面的一些实施例，在一些实施例中，所述备选切换策略包括以下至少之一：
第一策略，所述第一策略为：基于无随机接入信道RACH-less的最近卫星切换策略；
第二策略，所述第二策略为：不基于RACH-less的最近卫星切换策略；
第三策略，所述第三策略为：基于RACH-less以及卫星可见度的切换策略；
第四策略，所述第四策略为：不基于RACH-less但基于卫星可见度的切换策略；
第五策略，所述第五策略为：基于RACH-less以及载波干扰和噪声比CINR的切换策略；
第六策略，所述第六策略为：不基于RACH-less但基于CINR的切换策略。
结合第一方面的一些实施例，在一些实施例中，所述第一效果包括以下至少之一：
链路频谱效率大于第一值；
切换率小于第二值；
切换成功率大于第三值；
切换延迟小于第四值；
切换中断时间小于第五值；
多普勒频移小于第六值。
结合第一方面的一些实施例，在一些实施例中，所述第一模型输出的切换策略为：达到第一效果的备 选切换策略中效果最好的备选切换策略；或者
所述第一模型输出的切换策略为：达到第一效果的备选切换策略中的任一备选切换策略。
在上述实施例中，第一模型在基于第二设备的网络环境参数输出切换策略时，是将备选切换策略中的“在当前网络环境下切换时可达到第一效果的备选切换策略”作为输出，由此可确保第一模型输出的切换策略不仅适配于当前网络环境，还可达到较好的切换效果，从而提高了卫星切换时的切换性能。
结合第一方面的一些实施例，在一些实施例中，所述第二设备所处的NTN的网络环境参数包括以下至少之一：
至少一个第一卫星的相关参数；所述第一卫星为所述第二设备的可见卫星；
至少一个第二卫星的相关参数；所述第二卫星为所述第二设备当前通信的第三卫星的可见卫星。
结合第一方面的一些实施例，在一些实施例中，所述第一卫星的相关参数包括以下至少之一：
用于指示所述第一卫星的第一标识；
所述第二设备与所述第一卫星之间的距离；
所述第二设备与所述第一卫星之间的信号强度。
结合第一方面的一些实施例，在一些实施例中，所述第二卫星的相关参数包括以下至少之一：
用于指示所述第二卫星的第二标识；
所述第三卫星与所述第二卫星之间的距离；
所述第三卫星与所述第二卫星之间的信号强度。
结合第一方面的一些实施例，在一些实施例中，所述第一卫星、第二卫星分别包括以下至少之一：低地球轨道LEO卫星、中地球轨道MEO卫星、高椭圆轨道HEO卫星。
在上述实施例中，限定了网络环境参数具体包括哪些内容，以便第一设备可以成功确定出这些网络环境参数，并进一步基于这些网络环境参数来确定切换策略。
结合第一方面的一些实施例，在一些实施例中，所述第一设备包括任一卫星、第二设备、第三设备中的至少之一；
其中，所述第二设备包括终端或地面站；所述第三设备为不同于所述第二设备的地面设备。
结合第一方面的一些实施例，在一些实施例中，所述第一设备为任一卫星，所述确定第一参数，包括：
接收所述第二设备上报的至少一个第一卫星的相关参数；
所述第一设备基于实现确定至少一个第二卫星的相关参数。
结合第一方面的一些实施例，在一些实施例中，所述第一设备为第二设备，所述确定第一参数，包括：
所述第一设备基于实现确定至少一个第一卫星的相关参数；
接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
结合第一方面的一些实施例，在一些实施例中，所述第一设备为第三设备，所述确定第一参数，包括：
接收所述第二设备发送的至少一个第一卫星的相关参数；
接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
在上述实施例中，提供了具体如何确定网络环境参数的方法，以便第一设备可以采用该方法成功确定出网络环境参数，并进一步基于这些网络环境参数来确定切换策略。
结合第一方面的一些实施例，在一些实施例中，所述第一设备为任一卫星，所述方法还包括：
向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换。
结合第一方面的一些实施例，在一些实施例中，所述第一设备为第二设备，所述方法还包括：
基于所述第一模型输出的切换策略进行卫星切换；或者
向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，并接收所述第三卫星发送的切换策略，基于所述第三卫星发送的切换策略进行卫星切换；其中，所述第三卫星发送的切换策略与所述第一模型输出的切换策略相同或不同。
结合第一方面的一些实施例，在一些实施例中，所述第一设备为第三设备，所述方法还包括：
向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换；或者
向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，以使所述第三卫星确定最终的切换策略；其中，所述第三卫星确定的最终的切换策略与所述第一模型输出的切换策略相同或不同。
在上述实施例中，提出了具体如何基于第一模型输出的切换策略进行卫星切换的方法，以便后续可以成功基于第一模型所输出的切换策略来进行卫星切换。
结合第一方面的一些实施例，在一些实施例中，所述第一模型的模型算法包括以下算法中至少一个：Q-学习Q-Learning算法、深度Q-网络DQN算法、双深度Q-网络DDQN算法、行为Actor-评估Critic算法。
在上述实施例中，提供了第一模型的模型算法，以便基于这些算法来对第一模型进行训练，确保第一模型可以基于网络环境参数准确地输出切换策略，保证了切换策略确定的准确性。
第二方面，本公开实施例提出了一种第一设备，包括：
处理模块，用于确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
所述处理模块，还用于将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
结合第二方面的一些实施例，在一些实施例中，所述第一模型输出的切换策略为备选切换策略中达到第一效果的切换策略。
结合第二方面的一些实施例，在一些实施例中，所述备选切换策略包括以下至少之一：
第一策略，所述第一策略为：基于无随机接入信道RACH-less的最近卫星切换策略；
第二策略，所述第二策略为：不基于RACH-less的最近卫星切换策略；
第三策略，所述第三策略为：基于RACH-less以及卫星可见度的切换策略；
第四策略，所述第四策略为：不基于RACH-less但基于卫星可见度的切换策略；
第五策略，所述第五策略为：基于RACH-less以及载波干扰和噪声比CINR的切换策略；
第六策略，所述第六策略为：不基于RACH-less但基于CINR的切换策略。
结合第二方面的一些实施例，在一些实施例中，所述第一效果包括以下至少之一：
链路频谱效率大于第一值；
切换率小于第二值；
切换成功率大于第三值；
切换延迟小于第四值；
切换中断时间小于第五值；
多普勒频移小于第六值。
结合第二方面的一些实施例，在一些实施例中，所述第一模型输出的切换策略为：达到第一效果的备选切换策略中效果最好的备选切换策略；或者
所述第一模型输出的切换策略为：达到第一效果的备选切换策略中的任一备选切换策略。
结合第二方面的一些实施例，在一些实施例中，所述第二设备所处的NTN的网络环境参数包括以下至少之一：
至少一个第一卫星的相关参数；所述第一卫星为所述第二设备的可见卫星；
至少一个第二卫星的相关参数；所述第二卫星为所述第二设备当前通信的第三卫星的可见卫星。
结合第二方面的一些实施例，在一些实施例中，所述第一卫星的相关参数包括以下至少之一：
用于指示所述第一卫星的第一标识；
所述第二设备与所述第一卫星之间的距离；
所述第二设备与所述第一卫星之间的信号强度。
结合第二方面的一些实施例，在一些实施例中，所述第二卫星的相关参数包括以下至少之一：
用于指示所述第二卫星的第二标识；
所述第三卫星与所述第二卫星之间的距离；
所述第三卫星与所述第二卫星之间的信号强度。
结合第二方面的一些实施例，在一些实施例中，所述第一卫星、第二卫星分别包括以下至少之一：低 地球轨道LEO卫星、中地球轨道MEO卫星、高椭圆轨道HEO卫星。
结合第二方面的一些实施例，在一些实施例中，所述第一设备包括任一卫星、第二设备、第三设备中的至少之一；
其中，所述第二设备包括终端或地面站；所述第三设备为不同于所述第二设备的地面设备。
结合第二方面的一些实施例，在一些实施例中，所述第一设备为任一卫星，所述确定第一参数，包括：
接收所述第二设备上报的至少一个第一卫星的相关参数；
所述第一设备基于实现确定至少一个第二卫星的相关参数。
结合第二方面的一些实施例，在一些实施例中，所述第一设备为第二设备，所述确定第一参数，包括：
所述第一设备基于实现确定至少一个第一卫星的相关参数；
接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
结合第二方面的一些实施例，在一些实施例中，所述第一设备为第三设备，所述确定第一参数，包括：
接收所述第二设备发送的至少一个第一卫星的相关参数；
接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
结合第二方面的一些实施例，在一些实施例中，所述第一设备为任一卫星，所述方法还包括：
向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换。
结合第二方面的一些实施例，在一些实施例中，所述第一设备为第二设备，所述方法还包括：
基于所述第一模型输出的切换策略进行卫星切换；或者
向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，并接收所述第三卫星发送的切换策略，基于所述第三卫星发送的切换策略进行卫星切换；其中，所述第三卫星发送的切换策略与所述第一模型输出的切换策略相同或不同。
结合第二方面的一些实施例，在一些实施例中，所述第一设备为第三设备，所述方法还包括：
向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换；或者
向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，以使所述第三卫星确定最终的切换策略；其中，所述第三卫星确定的最终的切换策略与所述第一模型输出的切换策略相同或不同。
结合第二方面的一些实施例，在一些实施例中，所述第一模型的模型算法包括以下算法中至少一个：Q-学习Q-Learning算法、深度Q-网络DQN算法、双深度Q-网络DDQN算法、行为Actor-评估Critic算法。
第三方面，本公开实施例提出了通信设备，上述通信设备包括：一个或多个处理器；用于存储指令的一个或多个存储器；其中，上述处理器用于调用上述指令以使得上述通信设备执行如第一方面、第一方面的可选实现方式、第二方面、第二方面的可选实现方式所描述的方法。
第四方面，本公开实施例提出了通信系统，上述通信系统包括卫星、第二设备、第三设备中的至少之一；其中，上述卫星或者第二设备或者第三设备被配置为执行如第一方面和第一方面的可选实现方式所描述的方法。
第五方面，本公开实施例提出了存储介质，上述存储介质存储有指令，当上述指令在通信设备上运行时，使得上述通信设备执行如第一方面、第一方面的可选实现方式所描述的方法。
第六方面，本公开实施例提出了程序产品，包括计算机程序，上述计算机程序被通信设备执行时实现如第一方面、第一方面的可选实现方式所描述的方法。
第七方面，本公开实施例提出了计算机程序，当其在计算机上运行时，使得计算机执行如第一方面、第一方面的可选实现方式所描述的方法。
第八方面，本公开实施例提出了一种芯片或芯片系统，该芯片或芯片系统包括处理电路，被配置为执行根据上述第一方面、第一方面的可选实现方式所描述的方法。
可以理解地，上述终端、网络设备、通信设备、通信系统、存储介质、程序产品、计算机程序均用于执行本公开实施例所提出的方法。因此，其所能达到的有益效果可以参考对应方法中的有益效果，此处不再赘述。
本公开实施例提出了切换策略确定方法、通信设备、通信系统、存储介质。在一些实施例中，切换策略确定方法与信息处理方法、信息发送方法、信息接收方法等术语可以相互替换，通信装置与信息处理装置、信息发送装置、信息接收装置等术语可以相互替换，信息处理系统、通信系统、信息发送系统、信息接收系统等术语可以相互替换。
本公开实施例并非穷举，仅为部分实施例的示意，不作为对本公开保护范围的具体限制。在不矛盾的情况下，某一实施例中的每个步骤均可以作为独立实施例来实施，且各步骤之间可以任意组合，例如，在某一实施例中去除部分步骤后的方案也可以作为独立实施例来实施，且在某一实施例中各步骤的顺序可以任意交换，另外，某一实施例中的可选实现方式可以任意组合；此外，各实施例之间可以任意组合，例如，不同实施例的部分或全部步骤可以任意组合，某一实施例可以与其他实施例的可选实现方式任意组合。
在各本公开实施例中，如果没有特殊说明以及逻辑冲突，各实施例之间的术语和/或描述具有一致性，且可以互相引用，不同实施例中的技术特征根据其内在的逻辑关系可以组合形成新的实施例。
本公开实施例中所使用的术语只是为了描述特定实施例的目的，而并非作为对本公开的限制。
在本公开实施例中，除非另有说明，以单数形式表示的元素，如“一个”、“一种”、“该”、“上述”、“所述”、“前述”、“这一”等，可以表示“一个且只有一个”，也可以表示“一个或多个”、“至少一个”等。例如，在翻译中使用如英语中的“a”、“an”、“the”等冠词(article)的情况下，冠词之后的名词可以理解为单数表达形式，也可以理解为复数表达形式。
在本公开实施例中，“多个”是指两个或两个以上。
在一些实施例中，“至少一者(at least one of)”、“至少一项(at least one of)”、“至少一个(at least one of)”、“一个或多个(one or more)”、“多个(a plurality of)”、“多个(multiple)等术语可以相互替换。
本公开实施例中的如“A、B、C……中的至少一者”、“A和/或B和/或C……”等描述方式，包括了A、B、C……中任意一个单独存在的情况，也包括了A、B、C……中任意多个的任意组合情况，每种情况可以单独存在；例如，“A、B、C中的至少一者”包括单独A、单独B、单独C、A和B组合、A和C组合、B和C组合、A和B和C组合的情况；例如，A和/或B包括单独A、单独B、A和B的组合的情况。
在一些实施例中，“在一情况下A，在另一情况下B”、“响应于一情况A，响应于另一情况B”等记载方式，根据情况可以包括以下技术方案：与B无关地执行A，即，在一些实施例中A；与A无关地执行B，即，在一些实施例中B；A和B被选择性执行，即，在一些实施例中从A与B中选择执行；A和B都被执行，即，在一些实施例中A和B。当有A、B、C等更多分支时也类似上述。
本公开实施例中的“第一”、“第二”等前缀词，仅仅为了区分不同的描述对象，不对描述对象的位置、顺序、优先级、数量或内容等构成限制，对描述对象的陈述参见权利要求或实施例中上下文的描述，不应因为使用前缀词而构成多余的限制。例如，描述对象为“字段”，则“第一字段”和“第二字段”中“字段”之前的序数词并不限制“字段”之间的位置或顺序，“第一”和“第二”并不限制其修饰的“字段”是否在同一个消息中，也不限制“第一字段”和“第二字段”的先后顺序。再如，描述对象为“等级”，则“第一等级”和“第二等级”中“等级”之前的序数词并不限制“等级”之间的优先级。再如，描述对象的数量并不受序数词的限制，可以是一个或者多个，以“第一装置”为例，其中“装置”的数量可以是一个或者多个。此外，不同前缀词修饰的对象可以相同或不同，例如，描述对象为“装置”，则“第一装置”和“第二装置”可以是相同的装置或者不同的装置，其类型可以相同或不同；再如，描述对象为“信息”，则“第一信息”和“第二信息”可以是相同的信息或者不同的信息，其内容可以相同或不同。
在一些实施例中，“包括A”、“包含A”、“用于指示A”、“携带A”，可以解释为直接携带A，也可以解释为间接指示A。
在一些实施例中，“响应于……”、“响应于确定……”、“在……的情况下”、“在……时”、“当……时”、“若……”、“如果……”等术语可以相互替换。
在一些实施例中，“大于”、“大于或等于”、“不小于”、“多于”、“多于或等于”、“不少于”、“高于”、“高于或等于”、“不低于”、“以上”等术语可以相互替换，“小于”、“小于或等于”、“不大于”、“少于”、“少于或等于”、“不多于”、“低于”、“低于或等于”、“不高于”、“以下”等术语可以相互替换。
在一些实施例中，装置等可以解释为实体的、也可以解释为虚拟的，其名称不限定于实施例中所记载 的名称，“装置”、“设备(equipment)”、“设备(device)”、“电路”、“网元”、“节点”、“功能”、“单元”、“部件(section)”、“系统”、“网络”、“芯片”、“芯片系统”、“实体”、“主体”等术语可以相互替换。
在一些实施例中，“网络”可以解释为网络中包含的装置(例如，接入网设备、核心网设备等)。
在一些实施例中，“接入网设备(access network device，AN device)”、“无线接入网设备(radio access network device，RAN device)”、“基站(base station，BS)”、“无线基站(radio base station)”、“固定台(fixed station)”、“节点(node)”、“接入点(access point)”、“发送点(transmission point，TP)”、“接收点(reception point，RP)”、“传输接收点(transmission/reception point，TRP)”、“面板(panel)”、“天线面板(antenna panel)”、“天线阵列(antenna array)”、“小区(cell)”、“宏小区(macro cell)”、“小型小区(small cell)”、“毫微微小区(femto cell)”、“微微小区(pico cell)”、“扇区(sector)”、“小区组(cell group)”、“载波(carrier)”、“分量载波(component carrier)”、“带宽部分(bandwidth part，BWP)”等术语可以相互替换。
在一些实施例中，“终端(terminal)”、“终端设备(terminal device)”、“用户设备(user equipment，UE)”、“用户终端(user terminal)”、“移动台(mobile station，MS)”、“移动终端(mobile terminal，MT)”、订户站(subscriber station)、移动单元(mobile unit)、订户单元(subscriber unit)、无线单元(wireless unit)、远程单元(remote unit)、移动设备(mobiledevice)、无线设备(wireless device)、无线通信设备(wireless communication device)、远程设备(remote device)、移动订户站(mobile subscriber station)、接入终端(access terminal)、移动终端(mobile terminal)、无线终端(wireless terminal)、远程终端(remote terminal)、手持设备(handset)、用户代理(user agent)、移动客户端(mobile client)、客户端(client)等术语可以相互替换。
在一些实施例中，接入网设备、核心网设备、或网络设备可以被替换为终端。例如，针对将接入网设备、核心网设备、或网络设备以及终端间的通信置换为多个终端间的通信(例如，也可以被称为设备对设备(device-to-device，D2D)、车联网(vehicle-to-everything，V2X)等)的结构，也可以应用本公开的各实施例。在该情况下，也可以设为终端具有接入网设备所具有的全部或部分功能的结构。此外，“上行”、“下行”等语言也可以被替换为与终端间通信对应的语言(例如，“侧行(side)”)。例如，上行信道、下行信道等可以被替换为侧行信道，上行链路、下行链路等可以被替换为侧行链路。
在一些实施例中，终端可以被替换为接入网设备、核心网设备、或网络设备。在该情况下，也可以设为接入网设备、核心网设备、或网络设备具有终端所具有的全部或部分功能的结构。
在一些实施例中，获取数据、信息等可以遵照所在地国家的法律法规。
在一些实施例中，可以在得到用户同意后获取数据、信息等。
此外，本公开实施例的表格中的每一元素、每一行、或每一列均可以作为独立实施例来实施，任意元素、任意行、任意列的组合也可以作为独立实施例来实施。
本公开中各表所示的对应关系可以被配置，也可以是预定义的。各表中的信息的取值仅仅是举例，可以配置为其他值，本公开并不限定。在配置信息与各参数的对应关系时，并不一定要求必须配置各表中示意出的所有对应关系。例如，本公开中的表格中，某些行示出的对应关系也可以不配置。又例如，可以基于上述表格做适当的变形调整，例如，拆分，合并等等。上述各表中标题示出参数的名称也可以采用通信装置可理解的其他名称，其参数的取值或表示方式也可以通信装置可理解的其他取值或表示方式。上述各表在实现时，也可以采用其他的数据结构，例如可以采用数组、队列、容器、栈、线性表、指针、链表、树、图、结构体、类、堆、散列表或哈希表等。
本公开中的预定义可以理解为定义、预先定义、存储、预存储、预协商、预配置、固化、或预烧制。
图1是根据本公开实施例示出的通信系统的架构示意图。如图1所示，通信系统100可以包括卫星、终端、地面站(如网关(Gate Way，GW))、第三设备。其中，终端可以接入卫星，卫星可以通过地面站接入数据网络，卫星通过数据网络可以为终端提供网络服务。以及，该第三设备可以为不同于终端或地面站的地面设备，该第三设备可以连接至终端或地面站，以与终端或地面站进行网络通信。可选地，该第三设备例如可以包括接入网设备和/或核心网设备。
在一些实施例中，终端例如包括手机(mobile phone)、可穿戴设备、物联网设备、具备通信功能的 汽车、智能汽车、平板电脑(Pad)、带无线收发功能的电脑、虚拟现实(virtual reality，VR)终端设备、增强现实(augmented reality，AR)终端设备、工业控制(industrial control)中的无线终端设备、无人驾驶(self-driving)中的无线终端设备、远程手术(remote medical surgery)中的无线终端设备、智能电网(smart grid)中的无线终端设备、运输安全(transportation safety)中的无线终端设备、智慧城市(smart city)中的无线终端设备、智慧家庭(smart home)中的无线终端设备中的至少一者，但不限于此。
在一些实施例中，接入网设备例如是将终端接入到无线网络的节点或设备，接入网设备可以包括5G通信系统中的演进节点B(evolved NodeB，eNB)、下一代演进节点B(next generation eNB，ng-eNB)、下一代节点B(next generation NodeB，gNB)、节点B(node B，NB)、家庭节点B(home node B，HNB)、家庭演进节点B(home evolved nodeB，HeNB)、无线回传设备、无线网络控制器(radio network controller，RNC)、基站控制器(base station controller，BSC)、基站收发台(base transceiver station，BTS)、基带单元(base band unit，BBU)、移动交换中心、6G通信系统中的基站、开放型基站(Open RAN)、云基站(Cloud RAN)、其他通信系统中的基站、无线保真(wireless fidelity，WiFi)系统中的接入节点中的至少一者，但不限于此。
在一些实施例中，本公开的技术方案可适用于Open RAN架构，此时，本公开实施例所涉及的接入网设备间或者接入网设备内的接口可变为Open RAN的内部接口，这些内部接口之间的流程和信息交互可以通过软件或者程序实现。
在一些实施例中，接入网设备可以由集中单元(central unit，CU)与分布式单元(distributed unit，DU)组成的，其中，CU也可以称为控制单元(control unit)，采用CU-DU的结构可以将接入网设备的协议层拆分开，部分协议层的功能放在CU集中控制，剩下部分或全部协议层的功能分布在DU中，由CU集中控制DU，但不限于此。
在一些实施例中，核心网设备可以是一个设备，包括一个或多个网元，也可以是多个设备或设备群，分别包括一个或多个网元中的全部或部分。网元可以是虚拟的，也可以是实体的。核心网例如包括演进分组核心(Evolved Packet Core，EPC)、5G核心网络(5G Core Network，5GCN)、下一代核心(Next Generation Core，NGC)中的至少一者。或者，该核心网设备也可以是一种位置管理功能网元。示例性地，位置管理功能网元包括位置服务器(location server)，位置服务器可以实现为以下任意一项：位置管理功能(Location Management Function，LMF)、增强服务的流动定位中心(Enhanced Serving Mobile Location Centre，E-SMLC)、安全用户平面定位(Secure User Plane Location，SUPL)和安全用户平面定位平台(SUPL Location Platform，SUPLLP)。
可以理解的是，本公开实施例描述的通信系统是为了更加清楚的说明本公开实施例的技术方案，并不构成对于本公开实施例提出的技术方案的限定，本领域普通技术人员可知，随着系统架构的演变和新业务场景的出现，本公开实施例提出的技术方案对于类似的技术问题同样适用。
下述本公开实施例可以应用于图1所示的通信系统100、或部分主体，但不限于此。图1所示的各主体是例示，通信系统可以包括图1中的全部或部分主体，也可以包括图1以外的其他主体，各主体数量和形态为任意，各主体之间的连接关系是例示，各主体之间可以不连接也可以连接，其连接可以是任意方式，可以是直接连接也可以是间接连接，可以是有线连接也可以是无线连接。
本公开各实施例可以应用于长期演进(Long Term Evolution，LTE)、LTE-Advanced(LTE-A)、LTE-Beyond(LTE-B)、SUPER 3G、IMT-Advanced、第四代移动通信系统(4th generation mobile communication system，4G))、第五代移动通信系统(5th generation mobile communication system，5G)、5G新空口(new radio，NR)、未来无线接入(Future Radio Access，FRA)、新无线接入技术(New-Radio Access Technology，RAT)、新无线(New Radio，NR)、新无线接入(New radio access，NX)、未来一代无线接入(Future generation radio access，FX)、Global System for Mobile communications(GSM(注册商标))、CDMA2000、超移动宽带(Ultra Mobile Broadband，UMB)、IEEE 802.11(Wi-Fi(注册商标))、IEEE 802.16(WiMAX(注册商标))、IEEE 802.20、超宽带(Ultra-WideBand，UWB)、蓝牙(Bluetooth(注册商标))、陆上公用移动通信网(Public Land Mobile Network，PLMN)网络、设备到设备(Device-to-Device，D2D)系统、机器到机器(Machine to Machine，M2M)系统、物联网(Internet of Things，IoT)系统、车联网(Vehicle-to-Everything，V2X)、利用其他切换策略确定方法的系统、基于它们而扩展的下一代系统等。 此外，也可以将多个系统组合(例如，LTE或者LTE-A与5G的组合等)应用。
图2A是根据本公开实施例示出的切换策略确定方法的交互示意图。如图2A所示，本公开实施例涉及切换策略确定方法，用于通信系统100，上述方法包括：
步骤2101、地面站确定至少一个第一卫星的相关参数。
可选地，在NTN网络中，卫星通常需要通过地面站连接至数据网络(Data Network)，以便通过该数据网络为卫星所连接的终端提供网络服务。
可选地，在一些实施例之中，该第一卫星可以为地面站的可见卫星，可选地，此处的“可见卫星”例如可以理解为：能够通信或能够连接或能够提供服务的卫星。
在一些实施例之中，该第一卫星的相关参数可以包括以下至少之一：
用于指示第一卫星的第一标识；
地面站与第一卫星之间的距离；
地面站与第一卫星之间的信号强度，该信号强度例如可以为信噪比(Signal to Noise Ratio，SNR)。
可选地，在一些实施例之中，该第一卫星可以包括以下至少之一：低地球轨道(Low Earth orbit，LEO)卫星、中地球轨道(Middle Earth Orbit，MEO)卫星、高椭圆轨道(Highly Elliptical Orbit，HEO)卫星。
步骤2102、第三卫星向地面站发送至少一个第二卫星的相关参数。
可选地，该第三卫星可以为地面站当前通信的卫星，或称为：地面站当前连接的卫星。
在一些实施例之中，该第二卫星可以为第三卫星的可见卫星，该第二卫星的相关参数可以包括以下至少之一：
用于指示第二卫星的第二标识；
第三卫星与第二卫星之间的距离；
第三卫星与第二卫星之间的信号强度，该信号强度例如可以为SNR。
可选地，在一些实施例之中，该第二卫星可以包括以下至少之一：LEO卫星、MEO卫星、HEO卫星。
可选地，在一些实施例之中，可以是地面站先向第三卫星请求至少一个第二卫星的相关参数，之后，再由第三卫星向地面站发送至少一个第二卫星的相关参数。
可选地，在一些实施例之中，上述的步骤2101与步骤2102之间的执行顺序无先后之分，在一些实施例之中，步骤2101与步骤2102可以同时执行，或者，步骤2101可以在步骤2102之前或之后执行。
步骤2103、地面站将至少一个第一卫星的相关参数和/或至少一个第二卫星的相关参数输入至第一模型得到第一模型输出的切换策略。
可选地，该第一模型部署在地面站上，可以用于强化学习或者深度强化学习，该切换策略可以用于地面站进行卫星切换。
在一些实施例之中，第一模型输出的切换策略可以为备选切换策略中达到第一效果的切换策略。可选地，在一些实施例之中，上述的“至少一个第一卫星的相关参数和/或至少一个第二卫星的相关参数”可以统称为地面站所处的NTN的网络环境参数，其可以反映出地面站所处的NTN的网络环境，以及，第一模型基于输入的网络环境参数可以从备选切换策略中学习出(如强化学习或者深度强化学习出)效果较好的切换策略以输出。
可选地，该备选切换策略可以包括以下至少之一：
第一策略，第一策略为：基于无随机接入信道(Random Access Channel-less，RACH-less)的最近卫星切换策略；
第二策略，第二策略为：不基于RACH-less的最近卫星切换策略；
第三策略，第三策略为：基于RACH-less以及卫星可见度的切换策略；
第四策略，第四策略为：不基于RACH-less但基于卫星可见度的切换策略；
第五策略，第五策略为：基于RACH-less以及载波干扰和噪声比(carrier-to-interference-and-noise ratio，CINR)的切换策略；
第六策略，第六策略为：不基于RACH-less但基于CINR的切换策略。
可选地，上述的“RACH-less”可以是指：从源卫星切换至目标卫星时，省略随机接入流程，以此减 少切换过程中随机接入流程所消耗的时间，缩短切换中断时延。上述的“不基于RACH-less”可以是指：从源卫星切换至目标卫星时，不省略随机接入流程，在一些实施例之中，“不基于RACH-less”也可以称为“legacy handover”或其他名称，本公开对此不做具体限定。
可选地，上述的“最近卫星切换策略”可以是指：终端总是连接至最近的卫星。示例的，图2B是根据本公开实施例示出的最近卫星切换策略的流程示意图，如图2B所示，在时刻t(i)时，地面站的可见度范围内包括卫星#1、卫星#2、卫星#3，此时，卫星#2是距离地面站最近的卫星，则地面站可以是连接至卫星#2。可选地，由于卫星的移动性，在时刻t(i+1)时，地面站的最近卫星从卫星#2变更为卫星#3，此时，地面站可以从卫星#2切换至卫星#3，以使得地面站总是连接至最近卫星。
可选地，上述的“基于卫星可见度的切换策略”可以是指：伴随着卫星移动，当地面站当前连接的卫星即将移动至地面站可见度范围之外时，地面站从当前连接卫星切换至地面站当前的最大可见卫星。可选地，该“最大可见卫星”例如可以理解为：距离地面站最远的可见卫星，且在接下来的时刻，伴随着卫星移动，该卫星距离地面站的距离逐渐缩小，直至成为地面站的最近卫星后，该卫星距离地面站的距离又逐渐增大，并逐渐移动至地面站可见度范围之外。示例的，图2C是根据本公开实施例示出的基于卫星可见度的切换策略的流程示意图，如图2C所示，在时刻t(i)时，地面站的所连接的卫星为卫星#1，由于卫星的移动性，在时刻t(i+1)时，卫星#1即将移动至地面站可见度范围之外，此时，卫星#4为地面站的最大可见卫星，则地面站可以从卫星#1切换至卫星#4。
可选地，上述的“基于CINR的切换策略”可以是指：当地面站与当前连接的卫星之间的CINR满足第一条件时，则地面站进行卫星切换，切换至CINR高于预设阈值或者切换至CINR最好的可见卫星。可选地，该第一条件例如可以为CINR比最大参考电平降低3db。
可选地，上述的第一效果可以包括以下至少之一：
链路频谱效率大于第一值，如：切换后的链路频谱效率大于第一值；
切换率小于第二值；可选地，该切换率可以是指“切换次数”。
切换成功率大于第三值；
切换延迟小于第四值；
切换中断时间小于第五值；
多普勒频移小于第六值。
可选地，在一些实施例之中，第一模型输出的切换策略可以为：达到第一效果的备选切换策略中效果最好的备选切换策略；或者，在另一些实施例之中，第一模型输出的切换策略可以为：达到第一效果的备选切换策略中的任一备选切换策略。
则由上述内容可知，第一模型在基于地面站的网络环境参数输出切换策略时，是将备选切换策略中的“在当前网络环境下切换时可达到第一效果的备选切换策略”作为输出，由此可确保第一模型输出的切换策略不仅适配于当前网络环境，还可达到较好的切换效果，从而提高了卫星切换时的切换性能。
可选地，在一些实施例之中，上述的第一模型的模型算法可以包括以下算法中至少一个：Q-学习(Q-Learning)算法、深度Q-网络(Deep Q-Network，DQN)算法、双深度Q-网络(Double Deep Q-Network，DDQN)算法、行为评估(Actor-Critic)算法。
可选地，上述的“Q-Learning算法”为强化学习算法，上述的“DQN算法、DDQN算法、Actor-Critic算法”为深度强化学习算法。可选地，强化学习可以理解为是一种通过智能体(Agent)与环境的交互来学习最佳行为策略的方法；其中，智能体在环境中观察当前状态，采取行动并获得奖励或惩罚，然后根据这个反馈调整策略，强化学习的目标是通过与环境的交互获得最大的累积奖励，强化学习不需要标记的训练数据，而是通过试错来学习，强化学习的一个重要组成部分是马尔可夫决策过程(Markov Decision Process，MDP)，它定义了强化学习的环境模型和智能体的决策策略。而深度强化学习是将深度学习的感知能力和强化学习的决策能力相结合，通过训练多层神经网络(例如本实施例中的第一模型)来学习复杂的特征和模式，以便该神经网络可以直接根据输入的图像和/或数据得出最佳行为策略，并且，深度强化学习需要训练数据来对神经网络进行训练。
以下以“Q-Learning算法(即强化学习算法)”、“DQN算法(即深度强化学习算法)”为例介绍第一模型的模型方法。
在一些实施例之中，当第一模型的模型方法为“Q-Learning算法(即强化学习算法)”时，第一模型需要根据输入的网络环境参数并采用Q值计算公式计算出各个备选切换策略对应的Q值，其中，不同备选切换策略对应的Q值不同，当Q值越大时，说明该备选切换策略的切换效果(即前述的第一效果)越好，则第一模型可以基于Q值输出切换效果较好的备选切换策略。可选地，图2D是根据本公开实施例示出的Q值的计算公式。如图2D所示，包括三个公式，其中，第一个公式可以用于计算Q值，第二个公式和第三个公式可以用于表示相邻时刻的Q值之间的关系。可选地，图2D的公式中的Q(s,a)表示备选切换策略对应的Q值；r(s,a)、R(s,a)表示在当前状态s下执行备选切换策略对应的动作a的奖励，可选地，该r(s,a)、R(s,a)可以是参考输入的网络环境参数确定的，示例的，假设根据输入的网络环境参数确定在当前网络环境下利用备选切换策略进行卫星切换时其切换效果(即前述的第一效果)较好，则r(s,a)、R(s,a)越高；Qt-1r(s,a)、Qt-1(s,a)均表示当前时刻的Q值，如表示：未切换前的状态下的Q值，Qt(s,a)表示下一时刻的Q值，如表示：基于备选切换策略进行卫星切换后的状态下的Q值，α表示学习率因子，0≤α≤1，以及，图2D中的γ是折现因子，maxa’Q(s’,a’)、maxaQ(s’,a)表示在新的状态和新的行动下未来最大的奖励，可选地，此处的“新的状态”可以理解为：基于备选切换策略进行卫星切换后的状态，此处的“新的行动”可以理解为新的状态下的所有可能的行动。则第一模型基于图2D所示的公式进行MDP即可学习出效果较好的切换策略并输出。
在一些实施例之中，当第一模型的模型方法为“DQN算法(即深度强化学习算法)”时，第一模型无需采用公式计算，而可以根据输入的网络环境参数直接确定出各个备选切换策略对应的Q值，并基于Q值输出切换效果较好的备选切换策略。由此，第一模型需要先学习到Q值的计算方式，以便可以基于网络环境参数直接确定出备选切换策略对应的Q值。则需要对第一模型进行训练，以使该第一模型学习到Q值的计算方式。在一些实施例之中，可以先确定出样本集，该样本集可以包括地面站历史时刻所处的NTN的网络环境参数，以及该历史时刻下对应的某一切换策略对应的Q值，之后，将该历史时刻的网络环境参数输入至第一模型并确定出第一模型输出的Q值，再基于损失函数来调整第一模型的模型参数，直至损失函数收敛为止。可选地，图2E是根据本公开实施例示出的损失函数的计算公式。如图2E所示，损失函数中的TD与图2D中的TDt含义相同，损失函数中的r与图2D中的R(s,a)、r(s,a)含义相同，损失函数中的maxa’Q(s’,a’,θ’)与图2D中的maxa’Q(s’,a’)、maxaQ(s’,a)含义相同，损失函数中的Q(s,a,θ)与图2D中的Qt-1(s,a)、Qt-1r(s,a)含义相同，其中，θ为随机参数。
步骤2104、地面站执行第一操作以进行卫星切换。
可选地，在一些实施例之中，该第一操作可以为：直接基于第一模型输出的切换策略进行卫星切换。也即是，地面站自身具有决策权，其可以决策具体的切换策略，示例的，可以是当地面站的卫星切换由条件切换(Conditional Handover，CHO)触发时，地面站可以直接基于第一模型输出的切换策略进行卫星切换。
或者，在另一些实施例之中，该第一操作可以包括：向第三卫星上报第一模型输出的切换策略，并接收第三卫星发送的切换策略，之后，再基于第三卫星发送的切换策略进行卫星切换；其中，第三卫星发送的切换策略与第一模型输出的切换策略相同或不同。也即是，地面站不具有决策权，其需要基于第三卫星的指示进行切换，示例的，可以是当地面站自主确定要进行卫星切换时，其可以向第三卫星上报第一模型输出的切换策略，以便后续基于第三卫星发送的切换策略进行卫星切换。
则由上述内容可知，当地面站需要进行卫星切换时，地面站会利用第一模型基于地面站所处的NTN的网络环境参数确定对应的切换策略，以使得所确定出的切换策略是匹配于地面站的网络环境的，从而当地面站进行卫星切换时，会基于地面站所处的不同网络环境来针对性的采用合适的切换策略进行卫星切换，提高了卫星切换的灵活性，避免了切换资源的浪费，满足了切换业务的实时要求。并且，由于本公开实施例中的切换策略是基于地面站所处的网络环境确定的，与“卫星类型”无关，因此，本公开方法的切换策略不会受限于卫星类型，则可以实现不同类型的卫星之间的切换，从而保证了不同类型卫星的协同工作，确保了通信性能。
本公开实施例所涉及的切换策略确定方法可以包括步骤2101～步骤2104中的至少一者。例如，步骤2101可以作为独立实施例来实施，步骤2102可以作为独立实施例来实施，步骤2103可以作为独立实施例来实施，步骤2101+S2102可以作为独立实施例来实施，但不限于此。
在本实施方式或实施例中，在不矛盾的情况下，各步骤可以独立、任意组合或交换顺序，可选方式或可选例可以任意组合，且可以与其他实施方式或其他实施例的任意步骤之间进行任意组合。
图2F是根据本公开实施例示出的切换策略确定方法的交互示意图。如图2F所示，本公开实施例涉及切换策略确定方法，用于通信系统100，上述方法包括：
步骤2201、终端确定至少一个第一卫星的相关参数。
可选地，在一些实施例之中，该第一卫星可以为终端的可见卫星，可选地，此处的“可见卫星”例如可以理解为：能够通信或能够连接或能够提供服务的卫星。
在一些实施例之中，该第一卫星的相关参数可以包括以下至少之一：
用于指示第一卫星的第一标识；
终端与第一卫星之间的距离；
终端与第一卫星之间的信号强度，该信号强度例如可以为SNR。
可选地，在一些实施例之中，该第一卫星可以包括以下至少之一：LEO卫星、MEO卫星、HEO卫星。
步骤2202、第三卫星向终端发送至少一个第二卫星的相关参数。
可选地，该第三卫星可以为终端当前通信的卫星，或称为：终端当前连接的卫星。在一些实施例之中，该第二卫星可以为第三卫星的可见卫星，关于第二卫星的相关参数可以参考上述实施例描述。
可选地，在一些实施例之中，可以是终端先向第三卫星请求至少一个第二卫星的相关参数，之后，再由第三卫星向终端发送至少一个第二卫星的相关参数。
可选地，在一些实施例之中，上述的步骤2201与步骤2202之间的执行顺序无先后之分，在一些实施例之中，步骤2201与步骤2202可以同时执行，或者，步骤2201可以在步骤2202之前或之后执行。
步骤2203、终端将至少一个第一卫星的相关参数和/或至少一个第二卫星的相关参数输入至第一模型得到第一模型输出的切换策略。
关于步骤2202-2203的详细介绍可以参考上述图2A实施例描述。
步骤2204、终端执行第一操作以进行卫星切换。
可选地，在一些实施例之中，该第一操作可以为：直接基于第一模型输出的切换策略进行卫星切换。也即是，终端自身具有决策权，其可以决策具体的切换策略，示例的，可以是当终端的卫星切换由CHO触发时，终端可以直接基于第一模型输出的切换策略进行卫星切换。
或者，在另一些实施例之中，该第一操作可以包括：向第三卫星上报第一模型输出的切换策略，并接收第三卫星发送的切换策略，之后，再基于第三卫星发送的切换策略进行卫星切换；其中，第三卫星发送的切换策略与第一模型输出的切换策略相同或不同。也即是，终端不具有决策权，其需要基于第三卫星的指示进行切换，示例的，可以是当终端自主确定要进行卫星切换时，其可以向第三卫星上报第一模型输出的切换策略，以便后续基于第三卫星发送的切换策略进行卫星切换。
则由上述内容可知，当终端需要进行卫星切换时，终端会利用第一模型基于终端所处的NTN的网络环境参数确定对应的切换策略，以使得所确定出的切换策略是匹配于地面站的网络环境的，从而当终端进行卫星切换时，会基于终端所处的不同网络环境来针对性的采用合适的切换策略进行卫星切换，提高了卫星切换的灵活性，避免了切换资源的浪费，满足了切换业务的实时要求。并且，由于本公开实施例中的切换策略是基于终端所处的网络环境确定的，与“卫星类型”无关，因此，本公开方法的切换策略不会受限于卫星类型，则可以实现不同类型的卫星之间的切换，从而保证了不同类型卫星的协同工作，确保了通信性能。
本公开实施例所涉及的切换策略确定方法可以包括步骤2201～步骤2204中的至少一者。例如，步骤2201可以作为独立实施例来实施，步骤2202可以作为独立实施例来实施，步骤2203可以作为独立实施例来实施，步骤2201+S2202可以作为独立实施例来实施，但不限于此。
在本实施方式或实施例中，在不矛盾的情况下，各步骤可以独立、任意组合或交换顺序，可选方式或可选例可以任意组合，且可以与其他实施方式或其他实施例的任意步骤之间进行任意组合。
图2G是根据本公开实施例示出的切换策略确定方法的交互示意图。如图2G所示，本公开实施例涉 及切换策略确定方法，用于通信系统100，上述方法包括：
步骤2301、第二设备向部署了第一模型的卫星发送至少一个第一卫星的相关参数。
可选地，该第二设备可以为需要进行卫星切换的设备，该第二设备例如可以包括终端或地面站。在一些实施例之中，该第一卫星可以为第二设备的可见卫星。该第一卫星的相关参数可以包括以下至少之一：
用于指示第一卫星的第一标识；
第二设备与第一卫星之间的距离；
第二设备与第一卫星之间的信号强度，该信号强度例如可以为SNR。
可选地，上述的“部署了第一模型的卫星”可以为任一卫星，例如，其可以为第一卫星，也可以不为第一卫星。
可选地，在一些实施例之中，第二设备可以预先知晓哪些卫星部署了第一模型，则当第二设备需要进行卫星切换时，其可以将至少一个第一卫星的相关参数发送至部署了第一模型的卫星。
步骤2302、部署了第一模型的卫星确定至少一个第二卫星的相关参数。
可选地，该第二卫星可以为第三卫星的可见卫星，该第三卫星可以为第二设备当前通信的卫星，或称为：第二设备当前连接的卫星。在一些实施例之中，关于第二卫星的相关参数可以参考上述实施例描述。
在一些实施例之中，上述的“部署了第一模型的卫星”也可以为第二卫星或第三卫星，或者，也可以不为第二卫星或第三卫星。可选地，当“部署了第一模型的卫星”为第三卫星时，其可以基于自身可见的卫星自主确定出至少一个第二卫星的相关参数；当“部署了第一模型的卫星”不为第三卫星时，其可以基于卫星的星历信息先确定出第三卫星可见的第二卫星，之后再确定出至少一个第二卫星的相关参数。
可选地，在一些实施例之中，上述的步骤2301与步骤2302之间的执行顺序无先后之分，在一些实施例之中，步骤2301与步骤2302可以同时执行，或者，步骤2301可以在步骤2302之前或之后执行。
步骤2303、部署了第一模型的卫星将至少一个第一卫星的相关参数和/或至少一个第二卫星的相关参数输入至第一模型得到第一模型输出的切换策略。
关于步骤2303的详细介绍可以参考上述图2A实施例描述。
步骤2304、部署了第一模型的卫星向第二设备指示第一模型输出的切换策略。
步骤2305、第二设备基于第一模型输出的切换策略进行卫星切换。
由上述内容可知，当第二设备需要进行卫星切换时，部署了第一模型的卫星会利用第一模型基于第二设备所处的NTN的网络环境参数确定对应的切换策略，以使得所确定出的切换策略是匹配于第二设备的网络环境的，从而当第二设备进行卫星切换时，会基于第二设备所处的不同网络环境来针对性的采用合适的切换策略进行卫星切换，提高了卫星切换的灵活性，避免了切换资源的浪费，满足了切换业务的实时要求。并且，由于本公开实施例中的切换策略是基于第二设备所处的网络环境确定的，与“卫星类型”无关，因此，本公开方法的切换策略不会受限于卫星类型，则可以实现不同类型的卫星之间的切换，从而保证了不同类型卫星的协同工作，确保了通信性能。
本公开实施例所涉及的切换策略确定方法可以包括步骤2301～步骤2305中的至少一者。例如，步骤2301可以作为独立实施例来实施，步骤2302可以作为独立实施例来实施，步骤2303可以作为独立实施例来实施，步骤2301+S2302可以作为独立实施例来实施，但不限于此。
在本实施方式或实施例中，在不矛盾的情况下，各步骤可以独立、任意组合或交换顺序，可选方式或可选例可以任意组合，且可以与其他实施方式或其他实施例的任意步骤之间进行任意组合。
图2H是根据本公开实施例示出的切换策略确定方法的交互示意图。如图2H所示，本公开实施例涉及切换策略确定方法，用于通信系统100，上述方法包括：
步骤2401、第二设备向部署了第一模型的第三设备发送至少一个第一卫星的相关参数。
可选地，该第二设备可以为需要进行卫星切换的设备，该第二设备例如可以包括终端或地面站。在一些实施例之中，该第一卫星可以为第二设备的可见卫星。关于第一卫星的相关参数的详细介绍可以参考上述实施例描述。
可选地，上述的“部署了第一模型的第三设备”可以为不同于第二设备的地面设备，例如，可以为网络设备，如接入网设备和/或核心网设备。
可选地，在一些实施例之中，第二设备可以预先知晓部署了第一模型的第三设备是哪一设备，则当第二设备需要进行卫星切换时，其可以将至少一个第一卫星的相关参数发送至部署了第一模型的第三设备。
步骤2402、第三卫星向部署了第一模型的第三设备至少一个第二卫星的相关参数。
可选地，该第三卫星可以为第二设备当前通信的卫星，或称为：第二设备当前连接的卫星。该第二卫星可以为第三卫星的可见卫星，在一些实施例之中，关于第二卫星的相关参数可以参考上述实施例描述。
可选地，在一些实施例之中，第三卫星可以预先知晓部署了第一模型的第三设备是哪一设备，则当第三卫星确定第二设备需要进行卫星切换时，其可以将至少一个第二卫星的相关参数发送至部署了第一模型的第三设备。
可选地，在一些实施例之中，上述的步骤2401与步骤2402之间的执行顺序无先后之分，在一些实施例之中，步骤2401与步骤2402可以同时执行，或者，步骤2401可以在步骤2402之前或之后执行。
步骤2403、第三设备将至少一个第一卫星的相关参数和/或至少一个第二卫星的相关参数输入至第一模型得到第一模型输出的切换策略。
关于步骤2403的详细介绍可以参考上述图2A实施例描述。
步骤2404、第三设备执行第二操作以使得第二设备进行卫星切换。
可选地，在一些实施例之中，该第二操作例如可以为：第三设备直接向第二设备指示第一模型输出的切换策略，以使第二设备基于该切换策略进行卫星切换。
或者，在另一些实施例之中，该第二操作例如可以为：第三设备向第二设备当前通信的第三卫星上报第一模型输出的切换策略，以使第三卫星确定最终的切换策略，则第三卫星可以向第二设备指示该最终的切换策略，由此第二设备可以基于该切换策略进行卫星切换。其中，第三卫星确定的最终的切换策略与第一模型输出的切换策略相同或不同。
则由上述内容可知，当第二设备需要进行卫星切换时，部署了第一模型的第三设备会利用第一模型基于第二设备所处的NTN的网络环境参数确定对应的切换策略，以使得所确定出的切换策略是匹配于第二设备的网络环境的，从而当第二设备进行卫星切换时，会基于第二设备所处的不同网络环境来针对性的采用合适的切换策略进行卫星切换，提高了卫星切换的灵活性，避免了切换资源的浪费，满足了切换业务的实时要求。并且，由于本公开实施例中的切换策略是基于第二设备所处的网络环境确定的，与“卫星类型”无关，因此，本公开方法的切换策略不会受限于卫星类型，则可以实现不同类型的卫星之间的切换，从而保证了不同类型卫星的协同工作，确保了通信性能。
本公开实施例所涉及的切换策略确定方法可以包括步骤2401～步骤2404中的至少一者。例如，步骤2401可以作为独立实施例来实施，步骤2402可以作为独立实施例来实施，步骤2403可以作为独立实施例来实施，步骤2401+S2402可以作为独立实施例来实施，但不限于此。
在本实施方式或实施例中，在不矛盾的情况下，各步骤可以独立、任意组合或交换顺序，可选方式或可选例可以任意组合，且可以与其他实施方式或其他实施例的任意步骤之间进行任意组合。
图3是根据本公开实施例示出的切换策略确定方法的交互示意图。如图3所示，本公开实施例涉及切换策略确定方法，用于第一设备，上述方法包括：
步骤3101、确定第一参数。
步骤3102、将第一参数输入至第一模型，以得到第一模型输出的切换策略。
可选地，所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
可选地，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
可选地，所述第一模型输出的切换策略为备选切换策略中达到第一效果的切换策略。
可选地，所述备选切换策略包括以下至少之一：
第一策略，所述第一策略为：基于无随机接入信道RACH-less的最近卫星切换策略；
第二策略，所述第二策略为：不基于RACH-less的最近卫星切换策略；
第三策略，所述第三策略为：基于RACH-less以及卫星可见度的切换策略；
第四策略，所述第四策略为：不基于RACH-less但基于卫星可见度的切换策略；
第五策略，所述第五策略为：基于RACH-less以及载波干扰和噪声比CINR的切换策略；
第六策略，所述第六策略为：不基于RACH-less但基于CINR的切换策略。
可选地，所述第一效果包括以下至少之一：
链路频谱效率大于第一值；
切换率小于第二值；
切换成功率大于第三值；
切换延迟小于第四值；
切换中断时间小于第五值；
多普勒频移小于第六值。
可选地，所述第一模型输出的切换策略为：达到第一效果的备选切换策略中效果最好的备选切换策略；或者
所述第一模型输出的切换策略为：达到第一效果的备选切换策略中的任一备选切换策略。
可选地，所述第二设备所处的NTN的网络环境参数包括以下至少之一：
至少一个第一卫星的相关参数；所述第一卫星为所述第二设备的可见卫星；
至少一个第二卫星的相关参数；所述第二卫星为所述第二设备当前通信的第三卫星的可见卫星。
可选地，所述第一卫星的相关参数包括以下至少之一：
用于指示所述第一卫星的第一标识；
所述第二设备与所述第一卫星之间的距离；
所述第二设备与所述第一卫星之间的信号强度。
可选地，所述第二卫星的相关参数包括以下至少之一：
用于指示所述第二卫星的第二标识；
所述第三卫星与所述第二卫星之间的距离；
所述第三卫星与所述第二卫星之间的信号强度。
可选地，所述第一卫星、第二卫星分别包括以下至少之一：低地球轨道LEO卫星、中地球轨道MEO卫星、高椭圆轨道HEO卫星。
可选地，所述第一设备包括任一卫星、第二设备、第三设备中的至少之一；
其中，所述第二设备包括终端或地面站；所述第三设备为不同于所述第二设备的地面设备。
可选地，所述第一设备为任一卫星，所述确定第一参数，包括：
接收所述第二设备上报的至少一个第一卫星的相关参数；
所述第一设备基于实现确定至少一个第二卫星的相关参数。
可选地，所述第一设备为第二设备，所述确定第一参数，包括：
所述第一设备基于实现确定至少一个第一卫星的相关参数；
接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
可选地，所述第一设备为第三设备，所述确定第一参数，包括：
接收所述第二设备发送的至少一个第一卫星的相关参数；
接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
可选地，所述第一设备为任一卫星，所述方法还包括：
向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换。
可选地，所述第一设备为第二设备，所述方法还包括：
基于所述第一模型输出的切换策略进行卫星切换；或者
向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，并接收所述第三卫星发送的切换策略，基于所述第三卫星发送的切换策略进行卫星切换；其中，所述第三卫星发送的切换策略与所述第一模型输出的切换策略相同或不同。
可选地，所述第一设备为第三设备，所述方法还包括：
向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换；或者
向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，以使所述第三卫星确定最终的切换策略；其中，所述第三卫星确定的最终的切换策略与所述第一模型输出的切换策略相同或不同。
可选地，所述第一模型的模型算法包括以下算法中至少一个：Q-学习Q-Learning算法、深度Q-网络DQN算法、双深度Q-网络DDQN算法、行为Actor-评估Critic算法。
关于步骤3101-3102的详细介绍可以参考上述实施例描述。
本公开实施例所涉及的确定方法可以包括步骤S3101～步骤S3102中的至少一者。例如，步骤S3101可以作为独立实施例来实施，步骤S3102可以作为独立实施例来实施，步骤S3101～步骤S3102可以作为独立实施例来实施，但不限于此。
在本实施方式或实施例中，在不矛盾的情况下，各步骤可以独立、任意组合或交换顺序，可选方式或可选例可以任意组合，且可以与其他实施方式或其他实施例的任意步骤之间进行任意组合。
以下为对上述方法的示例性介绍。
本公开是在6G的异构网络中，采用深度强化学习技术，根据星座类型，即低地球轨道(LEO)，中地球轨道(MEO)和高椭圆轨道(HEO)，为每个星座确定最合适的HO策略，达到提高资源利用率并满足业务的实时性的需求。可选地，图4A是根据本公开实施例示出的切换策略确定方法的交互示意图。
在每个卫星上都进行深度强化学习训练。图4B是根据本公开实施例示出的深度学习方法的结构示意图。
将HO策略的过程，转化为马尔科夫决策过程:
马尔可夫决策过程是由5个数据成分(S,A,R,P，γ)组成的集合。
式中:S是状态的有限集合；A是一组有限的行动，A(s)是一组有限的活动，可以从各状态获得；
P是在时刻t处于状态s导致时刻t+1处于状态s'的概率；R为从执行动作a的状态切换状态s’后获得的直接奖励；γ是折现因子，它将表示未来奖励和当前奖励之间的关键差异。
马尔可夫决策过程的核心问题是找到一个“政策”，被称为(s)，当处于状态s时，哪一种行为选择模式可以获得最大化随机奖励的累积函数，图4C是根据本公开实施例示出的随机奖励的累积函数的计算公式。
Q-Learning
参考上述图2D的第一个公式，agent知道该选择哪种行动来获得最大的回报。
状态集s下的Q行动表a等于奖励r(s,a)，加上行动a时s的下一个最大的状态表。这一步创建了一个行动状态矩阵，这样每个智能体状态只需要找到具有最大表的行动。然而，强化学习是随机的，所以移动前后的表会有所不同。图4D是根据本公开实施例示出的Q-learning过程的示意图。因此,可以根据上述图2D所示公式以及图4D确定Q-learning过程。其中:TDt是计算瞬时值。α为学习率因子。
Deep Q-Network
另一种Q-learning方法，deep Q-network，是用神经网络代替动作状态Q表。图4E是根据本公开实施例示出的deep Q-network的神经网络示意图。
然而，需要深度Q-network方法来确定损失函数，以便神经网络学习如何估计表Q，从而使动作正确。损失函数必须计算表Q和Q现实与预测之间的误差。
根据上述图2E所示公式确定损失函数。
式中:θ为随机参数
设计一个DQN的agent
要估计策略的值，使用Q表函数近似器。因为DQN agent有自己的动作空间，所以可以使用多输出Q表函数。表Q向量接受单个观察作为输入，并返回具有尽可能多的元素和可行操作作为输出的单个向量。
当agent从状态匹配开始到提供的观察，并根据因素的数量执行动作时，每个输出元素的值表示折现的累积长期奖励。
奖励函数reward:为了同时保证各个业务在不同方面的性能，奖励函数依据链路频谱效率、HO率、HO 成功率，HO延迟、HO中断时间和多普勒频移这些方面进行设置。
DQN执行以下步骤:
状态集将一个状态引入网络；输出是单个动作的Q个表。
agent选择使用策略进行操作并执行该操作。
状态集返回状态s'，常规部分r是动作a的结果，并将[s,a,r,s']保存到内存中。
将实验样本分成几批进行神经网络训练。
该过程迭代到最后N个周期后。在每一个周期，HO类型(最近的卫星HO、最大能见度HO、基于cinr的HO)被允许做出决定，并多次接收状态变化的反馈，以自行获取信息并学习如何做出正确的决定。
该神经网络作为一个智能体进行经验训练，指示每周期迭代的步数和最大集训练次数可根据训练效果进行调整。
该算法根据输出对离散动作进行排序训练模型的。根据奖励函数中考虑的其他因素可以获得环境的状态，最后通过动作排序，选择一个最优的HO类型的动作。
同时，在训练HO类型的同时，也可以训练选择RACH-less handover还是legacy handover是最优的。
本公开实施例还提出用于实现以上任一方法的装置，例如，提出一装置，上述装置包括用以实现以上任一方法中终端所执行的各步骤的单元或模块。再如，还提出另一装置，包括用以实现以上任一方法中网络设备(例如接入网设备、核心网功能节点、核心网设备等)所执行的各步骤的单元或模块。
应理解以上装置中各单元或模块的划分仅是一种逻辑功能的划分，在实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。此外，装置中的单元或模块可以以处理器调用软件的形式实现：例如装置包括处理器，处理器与存储器连接，存储器中存储有指令，处理器调用存储器中存储的指令，以实现以上任一方法或实现上述装置各单元或模块的功能，其中处理器例如为通用处理器，例如中央处理单元(Central Processing Unit，CPU)或微处理器，存储器为装置内的存储器或装置外的存储器。或者，装置中的单元或模块可以以硬件电路的形式实现，可以通过对硬件电路的设计实现部分或全部单元或模块的功能，上述硬件电路可以理解为一个或多个处理器；例如，在一种实现中，上述硬件电路为专用集成电路(application-specific integrated circuit，ASIC)，通过对电路内元件逻辑关系的设计，实现以上部分或全部单元或模块的功能；再如，在另一种实现中，上述硬件电路为可以通过可编程逻辑器件(programmable logic device，PLD)实现，以现场可编程门阵列(Field Programmable Gate Array，FPGA)为例，其可以包括大量逻辑门电路，通过配置文件来配置逻辑门电路之间的连接关系，从而实现以上部分或全部单元或模块的功能。以上装置的所有单元或模块可以全部通过处理器调用软件的形式实现，或全部通过硬件电路的形式实现，或部分通过处理器调用软件的形式实现，剩余部分通过硬件电路的形式实现。
在本公开实施例中，处理器是具有信号处理能力的电路，在一种实现中，处理器可以是具有指令读取与运行能力的电路，例如中央处理单元(Central Processing Unit，CPU)、微处理器、图形处理器(graphics processing unit，GPU)(可以理解为微处理器)、或数字信号处理器(digital signal processor，DSP)等；在另一种实现中，处理器可以通过硬件电路的逻辑关系实现一定功能，上述硬件电路的逻辑关系是固定的或可以重构的，例如处理器为专用集成电路(application-specific integrated circuit，ASIC)或可编程逻辑器件(programmable logic device，PLD)实现的硬件电路，例如FPGA。在可重构的硬件电路中，处理器加载配置文档，实现硬件电路配置的过程，可以理解为处理器加载指令，以实现以上部分或全部单元或模块的功能的过程。此外，还可以是针对人工智能设计的硬件电路，其可以理解为ASIC，例如神经网络处理单元(Neural Network Processing Unit，NPU)、张量处理单元(Tensor Processing Unit，TPU)、深度学习处理单元(Deep learning Processing Unit，DPU)等。
图5是本公开实施例提出的第一设备的结构示意图。如图5所示，包括：
处理模块，用于确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
所述处理模块，还用于将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
可选地，上述处理模块用于执行以上任一方法中第一设备执行的与“处理”有关的步骤，上述第一设备还包括收发模块，该收发模块用于执行以上任一方法中第一设备执行的与“收发”有关的步骤。
图6A是本公开实施例提出的通信设备6100的结构示意图。通信设备6100可以是网络设备(例如接入网设备、核心网设备等)，也可以是终端(例如用户设备或上述的第一设备等)，也可以是支持网络设备实现以上任一方法的芯片、芯片系统、或处理器等，还可以是支持终端实现以上任一方法的芯片、芯片系统、或处理器等。通信设备6100可用于实现上述方法实施例中描述的方法，具体可以参见上述方法实施例中的说明。
如图6A所示，通信设备6100包括一个或多个处理器6101。处理器6101可以是通用处理器或者专用处理器等，例如可以是基带处理器或中央处理器。基带处理器可以用于对通信协议以及通信数据进行处理，中央处理器可以用于对通信装置(如，基站、基带芯片，终端设备、终端设备芯片，DU或CU等)进行控制，执行程序，处理程序的数据。处理器6101用于调用指令以使得通信设备6100执行以上任一方法。
在一些实施例中，通信设备6100还包括用于存储指令的一个或多个存储器6102。可选地，全部或部分存储器6102也可以处于通信设备6100之外。
在一些实施例中，通信设备6100还包括一个或多个收发器6103。在通信设备6100包括一个或多个收发器6103时，上述方法中的发送接收等通信步骤由收发器6103执行，其他步骤由处理器6101执行。
在一些实施例中，收发器可以包括接收器和发送器，接收器和发送器可以是分离的，也可以集成在一起。可选地，收发器、收发单元、收发机、收发电路等术语可以相互替换，发送器、发送单元、发送机、发送电路等术语可以相互替换，接收器、接收单元、接收机、接收电路等术语可以相互替换。
可选地，通信设备6100还包括一个或多个接口电路6104，接口电路6104与存储器6102连接，接口电路6104可用于从存储器6102或其他装置接收信号，可用于向存储器6102或其他装置发送信号。例如，接口电路6104可读取存储器6102中存储的指令，并将该指令发送给处理器6101。
以上实施例描述中的通信设备6100可以是网络设备或者终端，但本公开中描述的通信设备6100的范围并不限于此，通信设备6100的结构可以不受图6a的限制。通信设备可以是独立的设备或者可以是较大设备的一部分。例如所述通信设备可以是：1)独立的集成电路IC，或芯片，或，芯片系统或子系统；(2)具有一个或多个IC的集合，可选地，上述IC集合也可以包括用于存储数据，程序的存储部件；(3)ASIC，例如调制解调器(Modem)；(4)可嵌入在其他设备内的模块；(5)接收机、终端设备、智能终端设备、蜂窝电话、无线设备、手持机、移动单元、车载设备、网络设备、云设备、人工智能设备等等；(6)其他等等。
图6B是本公开实施例提出的芯片6200的结构示意图。对于通信设备6100可以是芯片或芯片系统的情况，可以参见图6B所示的芯片6200的结构示意图，但不限于此。
芯片6200包括一个或多个处理器6201，处理器6201用于调用指令以使得芯片6200执行以上任一方法。
在一些实施例中，芯片6200还包括一个或多个接口电路6202，接口电路6202与存储器6203连接，接口电路6202可以用于从存储器6203或其他装置接收信号，接口电路6202可用于向存储器6203或其他装置发送信号。例如，接口电路6202可读取存储器6203中存储的指令，并将该指令发送给处理器6201。可选地，接口电路、接口、收发管脚、收发器等术语可以相互替换。
在一些实施例中，芯片6200还包括用于存储指令的一个或多个存储器6203。可选地，全部或部分存储器6203可以处于芯片6200之外。
本公开还提出存储介质，上述存储介质上存储有指令，当上述指令在通信设备6100上运行时，使得通信设备6100执行以上任一方法。可选地，上述存储介质是电子存储介质。可选地，上述存储介质是计算机可读存储介质，但不限于此，其也可以是其他装置可读的存储介质。可选地，上述存储介质可以是非暂时性(non-transitory)存储介质，但不限于此，其也可以是暂时性存储介质。
本公开还提出程序产品，上述程序产品被通信设备6100执行时，使得通信设备6100执行以上任一方法。可选地，上述程序产品是计算机程序产品。
本公开还提出计算机程序，当其在计算机上运行时，使得计算机执行以上任一方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现 时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机程序。在计算机上加载和执行所述计算机程序时，全部或部分地产生按照本公开实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机程序可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机程序可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(digital video disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (22)

  1. 一种切换策略确定方法，其特征在于，由第一设备执行，所述方法包括：
    确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
    将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
  2. 如权利要求1所述的方法，其特征在于，所述第一模型输出的切换策略为备选切换策略中达到第一效果的切换策略。
  3. 如权利要求2所述的方法，其特征在于，所述备选切换策略包括以下至少之一：
    第一策略，所述第一策略为：基于无随机接入信道RACH-less的最近卫星切换策略；
    第二策略，所述第二策略为：不基于RACH-less的最近卫星切换策略；
    第三策略，所述第三策略为：基于RACH-less以及卫星可见度的切换策略；
    第四策略，所述第四策略为：不基于RACH-less但基于卫星可见度的切换策略；
    第五策略，所述第五策略为：基于RACH-less以及载波干扰和噪声比CINR的切换策略；
    第六策略，所述第六策略为：不基于RACH-less但基于CINR的切换策略。
  4. 如权利要求2或3所述的方法，其特征在于，所述第一效果包括以下至少之一：
    链路频谱效率大于第一值；
    切换率小于第二值；
    切换成功率大于第三值；
    切换延迟小于第四值；
    切换中断时间小于第五值；
    多普勒频移小于第六值。
  5. 如权利要求2-4任一所述的方法，其特征在于，所述第一模型输出的切换策略为：达到第一效果的备选切换策略中效果最好的备选切换策略；或者
    所述第一模型输出的切换策略为：达到第一效果的备选切换策略中的任一备选切换策略。
  6. 如权利要求1-5任一所述的方法，其特征在于，所述第二设备所处的NTN的网络环境参数包括以下至少之一：
    至少一个第一卫星的相关参数；所述第一卫星为所述第二设备的可见卫星；
    至少一个第二卫星的相关参数；所述第二卫星为所述第二设备当前通信的第三卫星的可见卫星。
  7. 如权利要求6所述的方法，其特征在于，所述第一卫星的相关参数包括以下至少之一：
    用于指示所述第一卫星的第一标识；
    所述第二设备与所述第一卫星之间的距离；
    所述第二设备与所述第一卫星之间的信号强度。
  8. 如权利要求6所述的方法，其特征在于，所述第二卫星的相关参数包括以下至少之一：
    用于指示所述第二卫星的第二标识；
    所述第三卫星与所述第二卫星之间的距离；
    所述第三卫星与所述第二卫星之间的信号强度。
  9. 如权利要求6-8任一所述的方法，其特征在于，所述第一卫星、第二卫星分别包括以下至少之一：低地球轨道LEO卫星、中地球轨道MEO卫星、高椭圆轨道HEO卫星。
  10. 如权利要求1-9任一所述的方法，其特征在于，所述第一设备包括任一卫星、第二设备、第三设备中的至少之一；
    其中，所述第二设备包括终端或地面站；所述第三设备为不同于所述第二设备的地面设备。
  11. 如权利要求10所述的方法，其特征在于，所述第一设备为任一卫星，所述确定第一参数，包括：
    接收所述第二设备上报的至少一个第一卫星的相关参数；
    所述第一设备基于实现确定至少一个第二卫星的相关参数。
  12. 如权利要求10所述的方法，其特征在于，所述第一设备为第二设备，所述确定第一参数，包括：
    所述第一设备基于实现确定至少一个第一卫星的相关参数；
    接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
  13. 如权利要求10所述的方法，其特征在于，所述第一设备为第三设备，所述确定第一参数，包括：
    接收所述第二设备发送的至少一个第一卫星的相关参数；
    接收所述第二设备当前通信的第三卫星发送的至少一个第二卫星的相关参数。
  14. 如权利要求10所述的方法，其特征在于，所述第一设备为任一卫星，所述方法还包括：
    向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换。
  15. 如权利要求10所述的方法，其特征在于，所述第一设备为第二设备，所述方法还包括：
    基于所述第一模型输出的切换策略进行卫星切换；或者
    向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，并接收所述第三卫星发送的切换策略，基于所述第三卫星发送的切换策略进行卫星切换；其中，所述第三卫星发送的切换策略与所述第一模型输出的切换策略相同或不同。
  16. 如权利要求10所述的方法，其特征在于，所述第一设备为第三设备，所述方法还包括：
    向所述第二设备指示所述第一模型输出的切换策略，以使所述第二设备基于所述切换策略进行卫星切换；或者
    向所述第二设备当前通信的第三卫星上报所述第一模型输出的切换策略，以使所述第三卫星确定最终的切换策略；其中，所述第三卫星确定的最终的切换策略与所述第一模型输出的切换策略相同或不同。
  17. 如权利要求1-16任一所述的方法，其特征在于，所述第一模型的模型算法包括以下算法中至少一个：Q-学习Q-Learning算法、深度Q-网络DQN算法、双深度Q-网络DDQN算法、行为Actor-评估Critic算法。
  18. 一种第一设备，其特征在于，包括：
    处理模块，用于确定第一参数；所述第一参数用于指示第二设备所处的非地面网络NTN的网络环境参数；所述第二设备为需进行卫星切换的设备；
    所述处理模块，还用于将所述第一参数输入至第一模型，以得到所述第一模型输出的切换策略；其中，所述第一模型用于进行强化学习或者深度强化学习，所述切换策略用于所述第二设备进行卫星切换。
  19. 一种通信设备，其特征在于，包括：
    一个或多个处理器；
    耦合于所述处理器上的存储器，所述存储器上存储有指令，当所述指令被所述处理器执行时，使所述通信设备执行权利要求1至17中任一项所述的方法。
  20. 一种通信系统，其特征在于，包括卫星、第二设备、第三设备中的至少之一，其中，所述卫星被配置为实现权利要求1-10、11、14、17中任一项所述的方法，所述第二设备被配置为实现权利要求1-10、12、15、17中任一项所述的方法，所述第三设备被配置为实现权利要求1-10、13、16、17中任一项所述的方法。
  21. 一种存储介质，所述存储介质存储有指令，其特征在于，当所述指令在通信设备上运行时，使得所述通信设备执行如权利要求1至17中任一项所述的方法。
  22. 一种程序产品，当所述程序产品其在通信设备上运行时，使得所述通信设备执行如权利要求1至17中任一项所述的信息处理方法。
PCT/CN2024/098126 2024-06-07 2024-06-07 切换策略确定方法、通信设备、通信系统、存储介质 Pending WO2025251297A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2024/098126 WO2025251297A1 (zh) 2024-06-07 2024-06-07 切换策略确定方法、通信设备、通信系统、存储介质
CN202480039750.1A CN121511630A (zh) 2024-06-07 2024-06-07 切换策略确定方法、通信设备、通信系统、存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2024/098126 WO2025251297A1 (zh) 2024-06-07 2024-06-07 切换策略确定方法、通信设备、通信系统、存储介质

Publications (1)

Publication Number Publication Date
WO2025251297A1 true WO2025251297A1 (zh) 2025-12-11

Family

ID=97959929

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/098126 Pending WO2025251297A1 (zh) 2024-06-07 2024-06-07 切换策略确定方法、通信设备、通信系统、存储介质

Country Status (2)

Country Link
CN (1) CN121511630A (zh)
WO (1) WO2025251297A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121664284A (zh) * 2026-01-29 2026-03-13 荣耀终端股份有限公司 卫星切换方法、通信系统、终端设备、卫星及相关装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113613301A (zh) * 2021-08-04 2021-11-05 北京航空航天大学 一种基于dqn的空天地一体化网络智能切换方法
US20220345967A1 (en) * 2021-04-19 2022-10-27 Xidian University Lightweight inter-satellite handover device and method for mega low-earth-orbit satellite networks
CN116803134A (zh) * 2020-10-30 2023-09-22 瑞典爱立信有限公司 非陆地网络中的切换命令
CN116916409A (zh) * 2023-08-21 2023-10-20 华能伊敏煤电有限责任公司 一种dqn辅助低轨卫星切换的决策生成方法
CN117296378A (zh) * 2023-08-03 2023-12-26 北京小米移动软件有限公司 失败处理方法及装置、存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116803134A (zh) * 2020-10-30 2023-09-22 瑞典爱立信有限公司 非陆地网络中的切换命令
US20220345967A1 (en) * 2021-04-19 2022-10-27 Xidian University Lightweight inter-satellite handover device and method for mega low-earth-orbit satellite networks
CN113613301A (zh) * 2021-08-04 2021-11-05 北京航空航天大学 一种基于dqn的空天地一体化网络智能切换方法
CN117296378A (zh) * 2023-08-03 2023-12-26 北京小米移动软件有限公司 失败处理方法及装置、存储介质
CN116916409A (zh) * 2023-08-21 2023-10-20 华能伊敏煤电有限责任公司 一种dqn辅助低轨卫星切换的决策生成方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121664284A (zh) * 2026-01-29 2026-03-13 荣耀终端股份有限公司 卫星切换方法、通信系统、终端设备、卫星及相关装置

Also Published As

Publication number Publication date
CN121511630A (zh) 2026-02-10

Similar Documents

Publication Publication Date Title
WO2025043684A1 (zh) 模型选择方法、节点及系统
WO2025007321A1 (zh) 省电处理方法、终端、网元
CN121511630A (zh) 切换策略确定方法、通信设备、通信系统、存储介质
CN115915170A (zh) 一种波束切换方法及装置
WO2024234164A1 (zh) 一种通信方法、设备及存储介质
WO2026036330A1 (zh) 一种通信方法及设备、通信系统、通信设备、存储介质
WO2026016145A1 (zh) Prach资源的状态指示方法及装置、存储介质
WO2026044689A1 (zh) 通信方法、设备及存储介质
CN122029871A (zh) 通信方法、终端、网络设备及存储介质
WO2026031095A1 (zh) 通信方法和装置、通信设备、通信系统及存储介质
WO2026011309A1 (zh) 移动边缘计算中的任务卸载和通信管理方法及装置、介质
WO2026065331A1 (zh) 通信方法、终端、网络设备及存储介质
WO2025138255A1 (zh) 通信方法、终端、网络设备及通信系统
WO2025081347A1 (zh) Ai模型处理方法、网络设备、用户设备及通信系统
WO2026036334A1 (zh) 一种通信方法及设备、通信系统、通信设备、存储介质
WO2025208452A1 (zh) 通信方法、装置以及存储介质
WO2025020085A1 (zh) 通信方法、中继器、网络设备以及存储介质
WO2025091433A1 (zh) 信息处理方法、终端、网络设备、通信系统及存储介质
WO2026081140A1 (zh) 资源配置方法、通信设备、通信系统及存储介质
WO2026011310A1 (zh) 通信方法、终端、网络设备以及系统和存储介质
WO2026044797A1 (zh) 通信方法、设备和存储介质
WO2025129453A1 (zh) 定位方法、装置及存储介质
WO2026031116A1 (zh) 通信方法、设备、系统及存储介质
WO2025231591A1 (zh) 通信方法、终端、网络设备、通信装置、通信系统及介质
WO2025260293A1 (zh) 通信方法、通信设备、通信系统、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24942215

Country of ref document: EP

Kind code of ref document: A1