CN116546579A

CN116546579A - Beam switching method, device and processor readable storage medium

Info

Publication number: CN116546579A
Application number: CN202210095304.5A
Authority: CN
Inventors: 索士强; 秦海超; 黄秋萍; 苏昕
Original assignee: Datang Mobile Communications Equipment Co Ltd
Current assignee: Datang Mobile Communications Equipment Co Ltd
Priority date: 2022-01-26
Filing date: 2022-01-26
Publication date: 2023-08-04
Also published as: WO2023143200A1

Abstract

The application provides a beam switching method, which comprises the following steps: acquiring first quality information corresponding to a plurality of first beams respectively in a first time period, wherein the first quality information corresponding to the plurality of first beams respectively comprises first quality information sent to a first network node by at least one User Equipment (UE); determining first average channel quality corresponding to the plurality of first beams based on first quality information corresponding to the plurality of first beams respectively; determining a first state vector based on first average channel qualities corresponding to the plurality of first beams and second average channel qualities corresponding to the plurality of second beams in at least one second time period; at least one second time period precedes the first time period; based on the first state vector, a beam switching threshold corresponding to at least one UE is determined. The method realizes the dynamic adjustment of the beam switching threshold value corresponding to at least one UE, and ensures proper beam switching frequency.

Description

Beam switching method, device and processor readable storage medium

Technical Field

The present application relates to the field of wireless communications technologies, and in particular, to a beam switching method, a beam switching device, and a processor readable storage medium.

Background

In the prior art, a channel between a network node and a UE (User Equipment) may be changed due to an obstacle or rotational movement of the UE, so that the quality of a beam between the network node and the UE is reduced or the quality of other beams is better than the quality of the beam; in this case, beam switching is required between the network node and the UE in order to obtain better system performance. In the switching process based on the fixed switching threshold, if the fixed switching threshold is smaller, although the better system performance can be maintained, frequent switching and even unnecessary switching can be caused, and load overhead is increased; if the fixed switching threshold is larger, although the switching frequency can be reduced, some switching cannot be triggered in time, so that the performance of the system cannot be guaranteed.

Disclosure of Invention

The present application provides a beam switching method, a beam switching device, and a processor readable storage medium for solving the above technical drawbacks.

In a first aspect, a beam switching method is provided, performed by a first network node, comprising:

acquiring first quality information corresponding to a plurality of first beams respectively in a first time period, wherein the first quality information corresponding to the plurality of first beams respectively comprises first quality information sent to a first network node by at least one User Equipment (UE);

Determining first average channel quality corresponding to the plurality of first beams based on first quality information corresponding to the plurality of first beams respectively;

determining a first state vector based on first average channel qualities corresponding to the plurality of first beams and second average channel qualities corresponding to the plurality of second beams in at least one second time period; at least one second time period precedes the first time period;

based on the first state vector, a beam switching threshold corresponding to at least one UE is determined.

In one embodiment, obtaining first quality information corresponding to each of the plurality of first beams in the first period of time includes:

acquiring first quality information sent by a second network node in a first time period; and receiving first quality information sent by at least one UE;

wherein the first quality information includes at least one of a signal-to-interference-and-noise ratio SINR, a received power RSRP, and a received quality RSRQ.

In one embodiment, determining the first state vector based on the first average channel quality for the plurality of first beams and the second average channel quality for the plurality of second beams over at least one second time period comprises:

carrying out quantization mapping processing on the first average channel quality to obtain a value of a first information vector corresponding to the first average channel quality; carrying out quantization mapping processing on the second average channel quality to obtain a value of a second information vector corresponding to the second average channel quality;

A first state vector is obtained based on the first information vector and the at least one second information vector.

In one embodiment, determining a beam switching threshold corresponding to at least one UE based on the first state vector includes:

inputting the first state vector into a first relation model, carrying out matching processing to obtain one or more reward values matched with the first state vector, and determining a beam switching threshold corresponding to the maximum reward value in the one or more reward values as the beam switching threshold corresponding to the UE;

wherein the first relationship model is used to characterize a relationship between the first state vector, the reward value, and the beam switching threshold.

In one embodiment, the first relationship model is determined by a first parameter corresponding to a first index change of at least one UE and a second parameter corresponding to a beam switching cost.

In one embodiment, the first relationship model is trained by:

constructing a training sample set; training the relation model based on the training sample set to obtain a first relation model;

based on the training sample set, training the relation model at least comprises:

inputting a second state vector in the training sample set to the relation model, and determining a beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

determining a loss function value based on the second state vector, the reward value corresponding to the second state vector and the loss function corresponding to the relation model;

and updating the model parameters of the relation model based on the loss function value to obtain an updated relation model.

In one embodiment, training the relational model based on the training sample set further comprises:

when the termination condition is not reached, the following steps are repeatedly performed:

inputting a second state vector in the training sample set to the updated relation model, and determining a beam switching threshold corresponding to the second state vector;

determining a loss function value based on the second state vector, the reward value corresponding to the second state vector and the loss function corresponding to the updated relationship model;

and updating the model parameters of the updated relationship model based on the loss function value to obtain the updated relationship model.

In one embodiment, after determining the loss function value based on the second state vector, the prize value corresponding to the second state vector, and the loss function corresponding to the relationship model, further comprising:

Judging whether a termination condition is reached;

if the termination condition is determined to be met, a first relation model is obtained;

the termination condition is one of the following:

the loss function value is less than or equal to the loss function value threshold; or alternatively, the process may be performed,

the loss function value is greater than or equal to the loss function value threshold.

In one embodiment, first quality information of at least one UE is sent to a first network node and to a second network node.

In a second aspect, a beam switching method is provided, which is performed by a UE and includes:

transmitting first quality information corresponding to a first wave beam in a first time period to a first network node; the method comprises the steps that a first network node obtains first quality information corresponding to a plurality of beams in a first time period respectively, and determines a beam switching threshold corresponding to UE (user equipment) based on the first quality information corresponding to the beams in at least one second time period and average channel quality corresponding to the beams;

and receiving a beam switching threshold value corresponding to the UE sent by the first network node, and performing beam switching based on the beam switching threshold value.

In a third aspect, a beam switching apparatus is provided, applied to a first network node, including a memory, a transceiver, and a processor:

A memory for storing a computer program; a transceiver for transceiving data under the control of the processor; a processor for reading the computer program in the memory and performing the following operations:

In a fourth aspect, a beam switching apparatus is provided, applied to a UE, including a memory, a transceiver, and a processor:

In a fifth aspect, the present application provides a beam switching apparatus, applied to a first network node, including:

the first processing unit is configured to obtain first quality information corresponding to each of a plurality of first beams in a first period, where the first quality information corresponding to each of the plurality of first beams includes first quality information sent by at least one user equipment UE to a first network node;

the second processing unit is used for determining first average channel quality corresponding to the plurality of first beams based on the first quality information corresponding to the plurality of first beams respectively;

a third processing unit, configured to determine a first state vector based on first average channel qualities corresponding to the plurality of first beams and second average channel qualities corresponding to the plurality of second beams in at least one second time period; at least one second time period precedes the first time period;

And the fourth processing unit is used for determining a beam switching threshold corresponding to at least one UE based on the first state vector.

In a sixth aspect, the present application provides a beam switching apparatus, applied to a UE, including:

a fifth processing unit, configured to send first quality information corresponding to a first beam in a first period of time to a first network node; the first network node obtains first quality information corresponding to the plurality of beams in the first time period, and determines a beam switching threshold corresponding to the UE based on the first quality information corresponding to the plurality of beams and the average channel quality corresponding to the plurality of beams in at least one second time period;

and the sixth processing unit is used for receiving the beam switching threshold value corresponding to the UE sent by the first network node and carrying out beam switching based on the beam switching threshold value.

In a seventh aspect, a processor readable storage medium is provided, wherein the processor readable storage medium stores a computer program for causing a processor to perform the methods of the first and second aspects.

The technical scheme provided by the embodiment of the application has at least the following beneficial effects:

the method and the device realize the dynamic adjustment of the beam switching threshold corresponding to at least one UE, so that the proper beam switching frequency can be ensured while the system performance can be maintained.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 is a schematic diagram of a system architecture provided in an embodiment of the present application;

fig. 2 is a flow chart of a beam switching method according to an embodiment of the present application;

fig. 3 is a schematic diagram of beam switching provided in an embodiment of the present application;

fig. 4 is a flow chart of another beam switching method according to an embodiment of the present application;

FIG. 5 is a schematic flow chart of reinforcement learning according to an embodiment of the present application;

fig. 6 is a flow chart of another beam switching method according to an embodiment of the present application;

fig. 7 is a flow chart of another beam switching method according to an embodiment of the present application;

fig. 8 is a schematic diagram of SINR statistics provided in an embodiment of the present application;

fig. 9 is a schematic diagram of SINR statistics provided in an embodiment of the present application;

fig. 10 is a schematic diagram of a switching count provided in an embodiment of the present application;

Fig. 11 is a schematic structural diagram of a beam switching device according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of a beam switching device according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of a beam switching device according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of a beam switching device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of illustrating the present application and are not to be construed as limiting the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.

In the embodiment of the application, the term "and/or" describes the association relationship of the association objects, which means that three relationships may exist, for example, a and/or B may be represented: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The term "plurality" in the embodiments of the present application means two or more, and other adjectives are similar thereto.

The term "determining B based on a" in the present application means that a is a factor to be considered in determining B. Not limited to "B can be determined based on A alone", it should also include: "B based on A and C", "B based on A, C and E", "C based on A, further B based on C", etc. Additionally, a may be included as a condition for determining B, for example, "when a satisfies a first condition, B is determined using a first method"; for another example, "when a satisfies the second condition, B" is determined, etc.; for another example, "when a satisfies the third condition, B" is determined based on the first parameter, and the like. Of course, a may be a condition in which a is a factor for determining B, for example, "when a satisfies the first condition, C is determined using the first method, and B is further determined based on C", or the like.

The term "determining B from a" in the present application means that a is considered when determining B. Not limited to "B can be determined from A alone", it should also include: "B determined from A and C", "B determined from A, C and E", "C determined from A, further B determined from C", etc. Additionally, a may be included as a condition for determining B, for example, "when a satisfies a first condition, B is determined using a first method"; for another example, "when a satisfies the second condition, B" is determined, etc.; for another example, "when a satisfies the third condition, B" is determined based on the first parameter, and the like. Of course, a may be a condition in which a is a factor for determining B, for example, "when a satisfies the first condition, C is determined using the first method, and B is further determined based on C", or the like.

In order to better understand and illustrate aspects of embodiments of the present disclosure, some technical terms related to the embodiments of the present disclosure are briefly described below.

(1) Beam switching technique

With the increasing demand for data traffic in communication systems, millimeter wave technology is becoming a key technology in 5G (5 th Generation Mobile Communication Technology, fifth generation mobile communication technology). While links of millimeter waves may suffer serious path loss due to high frequency characteristics of the millimeter waves, in order to overcome this disadvantage, millimeter wave communication systems need to use Massive MIMO (Multiple-input Multiple-output) technology based on beam forming. The beamforming technique can effectively improve the spectral efficiency of the mobile communication system, but each beam has a limited coverage area and requires beam switching to maintain good system performance. For example, when a UE leaves the coverage area of a serving beam, the serving beam will no longer fit into the UE, beam switching is required, or there is an obstacle between the network node and the UE such that the current serving beam quality is suddenly degraded, beam switching is also required.

In an actual application scenario, a network node, such as a millimeter wave base station, has a smaller coverage area, so that the deployment density of the millimeter wave base station is higher than that of an LTE (Long Term Evolution ) base station. In a millimeter wave network using beams, beam switching occurs not only between millimeter wave base stations but also between beams inside the same millimeter wave base station. The dense deployment of millimeter wave base stations and the use of beams will further increase the frequency of beam switching, which may limit the performance of the actual system, so the beam switching process needs to be optimized.

During beam management, beam scanning and reporting are performed periodically, and beam switching may be performed to obtain better communication performance when the quality of the current serving beam fluctuates based on the results of beam reporting. For this type of beam switching, the parameters of the switching can be optimized to improve system performance and reduce the probability of beam failure, while reducing the number of beam switching to reduce the load overhead of the system.

(2) Beam switching based on fixed threshold (fixed switching threshold)

The beam switch based on measurement reports comprises a two-part switch by a network node, e.g. a base station: one part is beam switching across the base station and the other part is beam switching inside the base station. Through practical simulation statistics, most beam switching occurs between different base stations. Therefore, referring to the A3 event in the cell switching mode, after the UE accesses a network node, such as an AP (Access Point), the UE periodically scans beams, stores the best N beam numbers and quality thereof after performing beam measurement on all the beams, and obtains candidate beams, and the UE feeds back related beam information (such as SINR (Signal to Interference plus Noise Ratio, signal to interference and noise ratio) to the AP, and the AP determines whether the beams are switched according to a fixed threshold determination method.

SINR is used as a reference index of beam quality; and reporting all the beam quality to trigger judgment of switching every time. Parameters related to the fixed threshold judgment method are as follows: TTT (Time-to-trigger), HM (Hysteresis Margin, hysteresis threshold), etc. HM represents the minimum difference between the SINR of the current serving beam and the SINR of the target beam, the threshold is denoted as delta, TTT represents the target beam satisfying the condition triggering the difference for a certain time. In a fixed threshold based handoff setting, if other beams are measured to be better quality than the serving beam and a certain threshold delta is exceeded, and the condition is maintained for a period of time (TTT), then a beam handoff is triggered. Switching judgmentThe constant flow process comprises the following steps: based on the reported beam quality, the AP judges whether the candidate beam enters the judging process of beam switching, if so, the SINR is satisfied _serve +Δ＜SINR _candidate If yes, entering a switching judgment, wherein the judgment process needs to last TTT time; after entering the switching judging process, if the SINR is satisfied in the TTT time _serve +Δ＜SINR _candidate If yes, then beam switching occurs after the judgment is finished, and the service beam is switched to the candidate beam; if the above condition is not satisfied at a certain time, the determination process is interrupted, and the determination is ended, and no beam switching occurs.

In the switching judging flow, selecting a beam with the largest SINR in each UE measuring result in a beam report as a candidate beam, and adding a candidate beam set; if the candidate beam is the service beam of other users, to avoid beam collision, the beam with the second largest SINR is selected as the candidate beam, and then the candidate beam is pushed. And triggering a switching judging process if the candidate wave beam meets the requirement of the threshold value. In the beam switching judging process, the candidate beam is not selected by other users, and the user does not select other beams to conduct switching judgment until one beam switching judgment is finished.

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

A schematic diagram of a system architecture provided in an embodiment of the present application is shown in FIG. 1, where the system architecture includes: UE, such as UE110, UE111, UE112, and UE113 in fig. 1, and network nodes, such as network node 120 and network node 121 in fig. 1. The network node is deployed in an access network, for example, an access network NG-RAN (New Generation-Radio Access Network, new Generation radio access network) in a 5G system. The UE and the network node communicate with each other via some kind of air interface technology, e.g. via cellular technology.

The UE according to the embodiments of the present application may be a device that provides voice and/or data connectivity to a user, a handheld device with a wireless connection function, or other processing device connected to a wireless modem, etc. Types of UEs include cell phones, vehicle user terminals, tablet computers, laptops, personal digital assistants, mobile internet appliances, wearable devices, and the like.

The network node to which the embodiments of the present application relate may be a base station, which may include a plurality of cells serving a terminal. Depending on the particular application, a base station may also be referred to as an access point, AP, or may be a device in an access network that communicates over the air-interface, through one or more sectors, with wireless terminal devices, or other names. The network node is operable to exchange received air frames with internet protocol (Internet Protocol, IP) packets as a router between the wireless terminal device and the rest of the access network, which may comprise an Internet Protocol (IP) communication network. The network node may also coordinate attribute management for the air interface. For example, the network node according to the embodiments of the present application may be a network device (Base Transceiver Station, BTS) in a global system for mobile communications (Global System for Mobile communications, GSM) or a code division multiple access (Code Division Multiple Access, CDMA), a network device (NodeB) in a wideband code division multiple access (Wide-band Code Division Multiple Access, WCDMA), an evolved network device (evolutional Node B, eNB or e-NodeB) in a long term evolution (long term evolution, LTE) system, a 5G base station (gNB) in a 5G network architecture (next generation system), a 6G base station in a super 5G mobile communication system (B5G), a 6G (6 th Generation Mobile Communication Technology, sixth generation mobile communication technology) network architecture, or a home evolved base station (Home evolved Node B, heNB), a relay node (relay node), a home base station (femto), a pico base station (pico), a base station, etc., which are not limited in the embodiments of the present application. In some network structures, the network nodes may include Centralized Unit (CU) nodes and Distributed Unit (DU) nodes, which may also be geographically separated.

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

In an embodiment of the present application, a beam switching method is provided and executed by a first network node, where a flow chart of the method is shown in fig. 2, and the method includes:

s201, first quality information corresponding to a plurality of first beams in a first time period is obtained, wherein the first quality information corresponding to the plurality of first beams comprises first quality information sent to a first network node by at least one User Equipment (UE).

Specifically, a first network node acquires first quality information corresponding to a plurality of beams of at least one second network node in a first time period; and receiving first quality information sent by at least one UE; the first quality information may include at least one of SINR, RSRP (Reference Signal Receiving Power, received power), RSRQ (Reference Signal Receiving Quality, received quality), among others. For example, the first network node obtains first quality information corresponding to the 20 first beams respectively in the first period, that is, in the first period, the first network node obtains 20 first quality information, where the 20 first quality information includes 10 first quality information sent by one or more second network nodes to the first network node, and 10 first quality information sent by the 10 UEs to the first network node respectively.

S202, determining first average channel quality corresponding to the first beams based on the first quality information corresponding to the first beams.

Specifically, for example, the first network node obtains 20 pieces of first quality information, the first quality information is SINR, the 20 pieces of first quality information is 20 SINR, an average value of the 20 SINR is calculated, and the average value of the 20 SINR is taken as the first average channel quality.

S203, determining a first state vector based on first average channel quality corresponding to the first beams and second average channel quality corresponding to the second beams in at least one second time period; the at least one second time period precedes the first time period.

Specifically, as shown in fig. 3, the sliding window includes N time windows, where the N time windows are a first time window, a second time window, a third time window, and an nth time window …, and the time lengths of the first time window, the second time window, the third time window, and the nth time window … may be preset, and N is a positive integer. The first time period is a first time window, and the first time window is a current window; the plurality of second time periods may be a second time window, a third time window, and an nth time window of …, respectively, where the second time window, the third time window, and the nth time window of … are all history windows; the first time window average channel quality may be a first average channel quality, the second time window average channel quality may be a second average channel quality, the third time window average channel quality may be a second average channel quality, and the nth time window average channel quality of … may be a second average channel quality; the second time window average channel quality, the third time window average channel quality, … nth time window average channel quality may differ from one another; the first time window average channel quality may be determined by SINR ₁ The second time window average channel quality, the third time window average channel quality, and the N time window average channel quality of … can be indicated as SINR respectively ₂ 、SINR ₃ 、…SINR _N And (3) representing.

It should be noted that, taking into account the trend of system performance over time and avoiding some unnecessary beam switching in the future, the use of sliding windows may take into account the influence of historical experience.

S204, determining a beam switching threshold corresponding to at least one UE based on the first state vector.

In particular, the beam switching threshold may include TTT, HM, etc.

In the embodiment of the application, the beam switching threshold corresponding to at least one UE is dynamically adjusted, so that the proper beam switching frequency can be ensured while the system performance can be maintained.

In one embodiment, the first network node may obtain first quality information corresponding to a plurality of beams of at least one second network node in a first period of time; and receives first quality information transmitted by at least one UE.

Specifically, the values of the first information vector and the second information vector are both discrete values, and the values of the first information vector and the second information vector may be used to characterize a signal strength level, which may be 1, 2, 3, … m, etc., where m is a positive integer. For another example, the first information vector is b ₁ The plurality of second information vectors may be b ₂ 、b ₃ 、…b _n The first state vector is (b) ₁ ，b ₂ ，b ₃ ，…b _n )。

In particular, the first relationship model may be a table that may be used to characterize the relationship between the first state vector, the reward value, and the beam switch threshold. The reward value may be used to characterize the effect of the UE to perform beam switching based on the beam switching threshold corresponding to the UE. The maximum prize value may be used to characterize the best effect of the UE to perform beam switching based on the UE's corresponding beam switching threshold.

In particular, the first indicator may comprise at least one of a channel capacity, a signal to noise ratio, a block error rate, BLER.

In one embodiment, the first relation model may be a table, which may be determined by a first parameter corresponding to a channel capacity change of at least one UE and a second parameter corresponding to a beam switching cost.

Specifically, the first parameter corresponding to the channel capacity variation of the at least one UE may beThe second parameter corresponding to the beam switching cost may be beta _c *H _n Wherein->The beam switching threshold corresponding to the state vector results in +.>Capacity change of->Mean value, beta, of channel capacity C representing at least one UE _c Is a preset punishment factor, H _n The average number of handovers.

In one embodiment, the first relationship model is trained by:

judging whether a termination condition is reached;

the termination condition is one of the following:

Specifically, a second state vector in the training sample set is input into the relation model, and a beam switching threshold corresponding to the second state vector is determined; based on the beam switching threshold value corresponding to the second state vector, obtaining a reward value corresponding to the second state vector through a reward function; substituting the second state vector and the rewarding value corresponding to the second state vector into the loss function corresponding to the relation model to obtain a loss function value, and updating the model parameters of the relation model based on the loss function value. Until the termination condition is reached, the relationship model when the termination condition is reached is taken as a first relationship model.

Specifically, if the first index is the channel capacity, inputting a second state vector in the training sample set to the relation model, and determining a beam switching threshold corresponding to the second state vector based on a greedy strategy; the second state vector is used to characterize the average channel quality of the plurality of beams over each training period.

Based on the beam switching threshold value corresponding to the second state vector, obtaining a reward value corresponding to the second state vector through a reward function; wherein, the reward function is as shown in formula (1):

Wherein, the liquid crystal display device comprises a liquid crystal display device,the beam switching threshold corresponding to the second state vector results in +.>Capacity change of->Mean value, beta, of channel capacity C representing at least one UE _c Is a preset punishment factor, H _n The average number of handovers.

The first relationship model may be deployed in a network node.

Specifically, the first network node may share the first quality information as side information with other network nodes (e.g., one or more second network nodes); the side information is information that can be shared among the network nodes.

In an embodiment of the present application, another beam switching method is provided and is executed by a UE, where a flow chart of the method is shown in fig. 4, and the method includes:

s401, first quality information corresponding to a first wave beam in a first time period is sent to a first network node; the first network node obtains first quality information corresponding to the plurality of beams in a first time period, and determines a beam switching threshold corresponding to the UE based on the first quality information corresponding to the plurality of beams and average channel quality corresponding to the plurality of beams in at least one second time period.

S402, receiving a beam switching threshold corresponding to the UE sent by the first network node, and performing beam switching based on the beam switching threshold.

If the UE measures that the quality of the other beams than the serving beam is better than that of the serving beam (e.g., the first beam), beam switching may be triggered based on a beam switching threshold, so that the UE switches from the serving beam to the other beams.

In the embodiment of the application, the beam switching threshold corresponding to the UE is dynamically adjusted, so that the proper beam switching frequency can be ensured while the system performance can be maintained.

The beam switching method according to the above embodiment of the present application is described in full detail by the following embodiments:

in the embodiment of the present application, reinforcement learning in a beam switching scenario is provided, and a flow chart of the reinforcement learning is shown in fig. 5, including:

s501, a beam switching threshold is preset.

In particular, the beam switching threshold may include TTT, HM, etc.

S502, constructing a training sample set.

In particular, each state vector in the training sample set may be used to characterize the average channel quality of the multiple beams over a training period.

S503, presetting a reward function.

Specifically, a bonus function as shown in formula (1) is set.

S504, training a relation model based on the training sample set to obtain a trained relation model.

Specifically, the trained relationship model is a first relationship model. According to the Q-learning algorithm, a relationship model is trained based on a training sample set. The relationship model may be a table, which may be a Q-table in a Q-learning algorithm, which may be used to characterize the relationship between state vectors, prize values, and beam switching thresholds; inputting the state vector in the training sample set into a table, and determining a beam switching threshold corresponding to the state vector; based on the beam switching threshold value corresponding to the state vector, obtaining a reward value corresponding to the state vector through a reward function; substituting the state vector and the rewarding value corresponding to the state vector into the loss function corresponding to the table to obtain a loss function value, and updating the model parameters of the table based on the loss function value. The following steps are repeatedly performed: inputting the state vector in the training sample set into a table, and determining a beam switching threshold corresponding to the state vector; based on the beam switching threshold value corresponding to the state vector, obtaining a reward value corresponding to the state vector through a reward function; substituting the state vector and the rewarding value corresponding to the state vector into the loss function corresponding to the table to obtain a loss function value, and updating the model parameters of the table based on the loss function value. Until the termination condition is reached, taking the form when the termination condition is reached as a first form (a relation model after training); wherein the termination condition may be that the loss function value is less than or equal to the loss function value threshold.

S505, deploying the trained relation model in the network node.

Specifically, the trained relationship model may be a first table, which is deployed in the network node.

In an embodiment of the present application, a method for switching a beam is provided, and a flow chart of the method is shown in fig. 6, where the method includes:

s601, the network node transmits a reference signal for beam scanning to the UE.

Specifically, the reference signal may be SSB (Synchronization Signal Block ).

S602, the UE performs beam measurement to obtain quality information corresponding to the service beam.

And S603, the UE feeds the quality information corresponding to the service beam and the quality information corresponding to the candidate beam back to the network node.

And S604, the network node takes the quality information corresponding to the service beam as side information, and shares the side information with other network nodes.

It should be noted that, if there are no other network nodes, step S604 is not required; the quality information corresponding to the service beam may be SINR, RSRP or RSRQ.

S605, the network node determines a state vector based on all the side information, and searches a beam switching threshold corresponding to the UE based on a relation model obtained by reinforcement learning training.

Specifically, the network node stores all the side information, where all the side information is the side information corresponding to the service beam, and the side information sent by other network nodes to the network node.

S606, the network node makes a beam switching decision based on the beam switching threshold; if the network node determines that the UE switches from the serving beam to the other beam, the network node instructs the UE to switch from the serving beam to the other beam, and proceeds to step S607 for execution.

In particular, the other beams may be candidate beams.

S607, the UE performs beam switching.

The embodiment of the application provides a beam switching method, which is applied to a multi-AP millimeter wave network scene, wherein the topology structure of the scene is as follows: uniformly deploying 3 APs and 7 users in a scene with a radius of 30 m; wherein 3 users of the 7 users move linearly along a certain path at a movement speed of 5km/h (kilometer/hour), each of the 3 APs has 32 beams, a hybrid beam forming technology is adopted, a period of beam scanning is set to 10ms, and when the UE satisfies a condition of beam switching, the switching is performed. A schematic flow chart of the method is shown in fig. 7, and the method comprises the following steps:

s701, the network node determines a beam switching threshold through a greedy strategy.

Specifically, the selectable option of beam switching threshold: (hm=1 dB and ttt=100 ms), (hm=3 dB and ttt=100 ms), (hm=5 dB and ttt=100 ms), (hm=7 dB and ttt=100 ms), and the like.

S702, the UE performs beam switching based on the beam switching threshold.

S703, the network node determines the second state vector by sliding a window.

For example, the sliding window includes 5 time windows, i.e., the length of the sliding window is 5, and the 5 time windows are a first time window, a second time window, a third time window, a fourth time window, and a fifth time window, respectively, the time length of the first time window is 10ms, and the time lengths of the second time window, the third time window, the fourth time window, and the fifth time window are all 40ms.

S704, the network node updates the Q table through the rewarding function.

Specifically, reinforcement learning training is performed according to a reward function shown in formula (1), namely training of a relation model, wherein the relation model is a Q table, and penalty factors beta in the reward function _c May be set to 0.75.

S705, the network node deploys the updated Q table into the network node; go to steps S701 and S706 to be performed, respectively.

S706, the network node determines a first state vector, and obtains a beam switching threshold value by looking up a Q table.

Specifically, the first state vector may be (b ₁ ，b ₂ ，b ₃ ，…b _n )。b ₁ ，b ₂ ，b ₃ ，…b _n The values of (2), 3, … m, etc., for example, the signal strength levels are 1, 2, 3, 4, and 5, i.e., the signal strength levels total 5 levels.

S707, the network node makes a beam switch decision.

S708, the network node performs periodic beam scanning.

S709, the UE performs beam measurement.

S710, the UE feeds back quality information corresponding to the wave beam; go to step S706 for execution.

Specifically, the quality information may be SINR, RSRP or RSRQ.

It should be noted that, steps S701-S705 are steps in the reinforcement learning training, and the update interval T of each reinforcement learning training may be 10ms (milliseconds); steps S706-S710 are steps in the online execution.

In one embodiment, as shown in fig. 8 and 9 (fig. 9 is an enlarged view of fig. 8), CDF (Cumulative Distribution Function ) statistics of SINR for a service beam over a time of 20s (seconds) based on a fixed threshold; the CDF statistical result of SINR corresponding to the service beam in 20s time is based on a beam switching method of reinforcement learning (such as Q-learning); the beam switching based on reinforcement learning is superior in performance.

In one embodiment, as shown in FIG. 10, the present application is based on beam switching for reinforcement learning (e.g., Q-learning) with minimal number of beam switching occurrences.

Based on the same inventive concept, the embodiments of the present application also provide a beam switching device applied to a first network node, where a schematic structure diagram of the device is shown in fig. 11, and a transceiver 1300 is used to receive and transmit data under the control of a processor 1310.

Where in FIG. 11, a bus architecture may comprise any number of interconnected buses and bridges, with one or more processors, represented by processor 1310, and various circuits of memory, represented by memory 1320, being linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. The bus interface provides an interface. Transceiver 1300 may be a number of elements, including a transmitter and a receiver, providing a means for communicating with various other apparatus over a transmission medium, including wireless channels, wired channels, optical cables, etc. The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 may store data used by the processor 1310 in performing operations.

The processor 1310 may be a Central Processing Unit (CPU), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA), or a complex programmable logic device (Complex Programmable Logic Device, CPLD), or the processor may employ a multi-core architecture.

A processor 1310 for reading the computer program in the memory and performing the following operations:

In one embodiment, the first relationship model is trained by:

judging whether a termination condition is reached;

the termination condition is one of the following:

It should be noted that, the above device provided in the embodiment of the present invention can implement all the method steps implemented in the method embodiment and achieve the same technical effects, and detailed descriptions of the same parts and beneficial effects as those in the method embodiment in this embodiment are omitted.

Based on the same inventive concept, the embodiments of the present application also provide a beam switching apparatus applied to a UE, the structure of which is schematically shown in fig. 12, and a transceiver 1400 for receiving and transmitting data under the control of a processor 1410.

Where in FIG. 12, a bus architecture may be comprised of any number of interconnected buses and bridges, and in particular one or more processors represented by the processor 1410 and various circuits of the memory represented by the memory 1420, are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. The bus interface provides an interface. Transceiver 1400 may be a number of elements, including a transmitter and a receiver, providing a means for communicating with various other apparatus over transmission media, including wireless channels, wired channels, optical cables, and the like. The user interface 1430 may also be an interface capable of interfacing with an inscribed desired device for a different user device, including but not limited to a keypad, display, speaker, microphone, joystick, etc.

The processor 1410 is responsible for managing the bus architecture and general processing, and the memory 1420 may store data used by the processor 1410 in performing operations.

Alternatively, the processor 1410 may be a CPU (central processing unit), ASIC (Application Specific Integrated Circuit ), FPGA (Field-Programmable Gate Array, field programmable gate array) or CPLD (Complex Programmable Logic Device ), and the processor may also employ a multi-core architecture.

The processor is configured to execute the method according to the second aspect provided in the embodiment of the present application by calling a computer program stored in the memory according to the obtained executable instructions. The processor and the memory may also be physically separate.

A processor 1410 for reading the computer program in the memory 1420 and performing the following operations:

Based on the same inventive concept as the foregoing embodiments, the present embodiment further provides a beam switching device applied to the first network node, where a schematic structure diagram of the beam switching device 80 is shown in fig. 13, and the beam switching device includes a first processing unit 801, a second processing unit 802, a third processing unit 803, and a fourth processing unit 804.

A first processing unit 801, configured to obtain first quality information corresponding to each of a plurality of first beams in a first period, where the first quality information corresponding to each of the plurality of first beams includes first quality information sent by at least one user equipment UE to a first network node;

a second processing unit 802, configured to determine first average channel qualities corresponding to the plurality of first beams based on first quality information corresponding to the plurality of first beams, respectively;

A third processing unit 803, configured to determine a first state vector based on first average channel qualities corresponding to the plurality of first beams and second average channel qualities corresponding to the plurality of second beams in at least one second time period; at least one second time period precedes the first time period;

a fourth processing unit 804 is configured to determine a beam switching threshold corresponding to at least one UE based on the first state vector.

In one embodiment, the first processing unit 801 is specifically configured to:

In one embodiment, the third processing unit 803 is specifically configured to:

In one embodiment, the fourth processing unit 804 is specifically configured to:

In one embodiment, the first relationship model is trained by:

Judging whether a termination condition is reached;

the termination condition is one of the following:

In one embodiment, the first processing unit 801 is further configured to:

and transmitting the first quality information of the at least one UE to the first network node to the second network node.

Based on the same inventive concept as the previous embodiments, the present embodiment further provides a beam switching device applied to a UE, where a schematic structure diagram of the beam switching device 90 is shown in fig. 14, and the beam switching device includes a fifth processing unit 901 and a sixth processing unit 902.

A fifth processing unit 901, configured to send first quality information corresponding to a first beam in a first period of time to a first network node; the first network node obtains first quality information corresponding to the plurality of beams in the first time period, and determines a beam switching threshold corresponding to the UE based on the first quality information corresponding to the plurality of beams and the average channel quality corresponding to the plurality of beams in at least one second time period;

The sixth processing unit 902 is configured to receive a beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice. In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Based on the same inventive concept, the embodiments of the present application further provide a processor readable storage medium storing a computer program for implementing the steps of any one of the embodiments or any one of the time determining methods for cell handover provided by any one of the alternative implementations of the embodiments of the present application when executed by a processor.

The processor-readable storage medium may be any available medium or data storage device that can be accessed by a processor including, but not limited to, magnetic memory (e.g., floppy disk, hard disk, tape, magneto-optical disk (MO), etc.), optical memory (e.g., CD, DVD, BD, HVD, etc.), and semiconductor memory (e.g., ROM, EPROM, EEPROM, nonvolatile memory (NAND FLASH), solid State Disk (SSD)), etc.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These processor-executable instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

Claims

1. A method of beam switching performed by a first network node, comprising:

acquiring first quality information corresponding to a plurality of first beams respectively in a first time period, wherein the first quality information corresponding to the plurality of first beams respectively comprises first quality information sent to the first network node by at least one User Equipment (UE);

Determining a first state vector based on first average channel quality corresponding to the plurality of first beams and second average channel quality corresponding to the plurality of second beams in at least one second time period; the at least one second time period precedes the first time period;

and determining a beam switching threshold corresponding to the at least one UE based on the first state vector.

2. The method of claim 1, wherein the obtaining the first quality information corresponding to each of the plurality of first beams in the first period of time includes:

acquiring first quality information sent by a second network node in a first time period; and receiving first quality information sent by the at least one UE;

wherein the first quality information includes at least one of a signal to interference and noise ratio SINR, a received power RSRP, and a received quality RSRQ.

3. The method of claim 1, wherein the determining the first state vector based on the first average channel quality for the plurality of first beams and the second average channel quality for the plurality of second beams for at least one second time period comprises:

carrying out quantization mapping processing on the first average channel quality to obtain a value of a first information vector corresponding to the first average channel quality; performing quantization mapping processing on the second average channel quality to obtain a value of a second information vector corresponding to the second average channel quality;

And obtaining a first state vector based on the first information vector and at least one second information vector.

4. The method of claim 1, wherein the determining the beam switching threshold corresponding to the at least one UE based on the first state vector comprises:

inputting the first state vector into a first relation model, performing matching processing to obtain one or more reward values matched with the first state vector, and determining a beam switching threshold corresponding to the maximum reward value in the one or more reward values as the beam switching threshold corresponding to the UE;

wherein the first relationship model is used to characterize a relationship between the first state vector, a reward value, and a beam switching threshold.

5. The method of claim 4, wherein the first relationship model is determined by a first parameter corresponding to a first index change of the at least one UE and a second parameter corresponding to a beam switch cost.

6. The method of claim 4, wherein the first relationship model is trained by:

constructing a training sample set; training a relationship model based on the training sample set to obtain the first relationship model;

The training of the relation model based on the training sample set at least comprises:

determining a reward value corresponding to the second state vector based on a beam switching threshold and a reward function corresponding to the second state vector;

determining a loss function value based on the second state vector, a reward value corresponding to the second state vector, and a loss function corresponding to the relationship model;

7. The method of claim 6, wherein the training a relationship model based on the set of training samples further comprises:

Determining a loss function value based on a second state vector, a reward value corresponding to the second state vector, and a loss function corresponding to the updated relationship model;

and updating the model parameters of the updated relation model based on the loss function value to obtain the updated relation model.

8. The method of claim 6, further comprising, after determining a loss function value based on the second state vector, the prize value for the second state vector, and the loss function for the relational model:

judging whether a termination condition is reached;

if the termination condition is determined to be met, obtaining the first relation model;

the termination condition is one of the following:

the loss function value is less than or equal to a loss function value threshold; or alternatively, the process may be performed,

the loss function value is greater than or equal to a loss function value threshold.

9. The method as recited in claim 1, further comprising:

and sending the first quality information of the at least one UE to the first network node to the second network node.

10. A method for beam switching performed by a UE, comprising:

Transmitting first quality information corresponding to a first wave beam in a first time period to a first network node; the first network node obtains first quality information corresponding to a plurality of beams in the first time period, and determines a beam switching threshold corresponding to the UE based on the first quality information corresponding to the plurality of beams and average channel quality corresponding to the plurality of beams in at least one second time period;

11. A beam switching device for a first network node, comprising a memory, a transceiver, and a processor:

a memory for storing a computer program; a transceiver for transceiving data under control of the processor; a processor for reading the computer program in the memory and performing the following operations:

12. A beam switching device, applied to a UE, comprising a memory, a transceiver, and a processor:

13. A beam switching apparatus for use in a first network node, comprising:

the first processing unit is configured to obtain first quality information corresponding to a plurality of first beams in a first period, where the first quality information corresponding to the plurality of first beams includes first quality information sent by at least one user equipment UE to the first network node;

a second processing unit, configured to determine first average channel qualities corresponding to the plurality of first beams based on first quality information corresponding to the plurality of first beams, respectively;

a third processing unit, configured to determine a first state vector based on first average channel qualities corresponding to the plurality of first beams and second average channel qualities corresponding to the plurality of second beams in at least one second time period; the at least one second time period precedes the first time period;

and the fourth processing unit is used for determining a beam switching threshold corresponding to the at least one UE based on the first state vector.

14. A beam switching apparatus for use in a UE, comprising:

a fifth processing unit, configured to send first quality information corresponding to a first beam in a first period of time to a first network node; the first network node obtains first quality information corresponding to a plurality of beams in the first time period, and determines a beam switching threshold corresponding to the UE based on the first quality information corresponding to the plurality of beams and average channel quality corresponding to the plurality of beams in at least one second time period;

And a sixth processing unit, configured to receive a beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

15. A processor-readable storage medium, characterized in that the processor-readable storage medium stores a computer program for causing the processor to perform the method of any one of claims 1 to 10.