WO2023143200A1 - Beam switching method and apparatus, and processor readable storage medium - Google Patents

Abstract

Provided in the present disclosure is a beam switching method, comprising: acquiring first quality information respectively corresponding to a plurality of first beams within a first time period, the first quality information respectively corresponding to the plurality of first beams comprising first quality information sent by at least one user equipment (UE) to a first network node; on the basis of the first quality information respectively corresponding to the plurality of first beams, determining a first average channel quality corresponding to the plurality of first beams; determining a first state vector on the basis of the first average channel quality corresponding to the plurality of first beams and a second average channel quality corresponding to a plurality of second beams within at least one second time period, the at least one second time period being earlier than the first time period; and on the basis of the first state vector, determining a beam switching threshold corresponding to the at least one UE. In the method, dynamic adjustment on a beam switching threshold corresponding to at least one UE is achieved, thereby ensuring a proper beam switching frequency.

Description

Beam switching method, device and processor-readable storage medium

Cross References to Related Applications

This disclosure claims the priority of Chinese Patent Application No. 202210095304.5 entitled "Beam Switching Method, Device, and Processor-Readable Storage Medium" filed at the State Intellectual Property Office of China on January 26, 2022, the entirety of which The contents are hereby incorporated by reference.

technical field

The present disclosure relates to the field of wireless communication technologies, and in particular, the present disclosure relates to a beam switching method, device, and processor-readable storage medium.

Background technique

In the prior art, the channel between the network node and UE (User Equipment, user equipment) may change due to obstacles or rotational movement of the UE, resulting in reduced beam quality between the network node and UE or quality of other beams than this Better beam quality; in this case, beam switching is required between network nodes and UEs for better system performance. In the handover process based on the fixed handover threshold, if the fixed handover threshold is small, although better system performance can be maintained, it will lead to frequent handover, even unnecessary handover, resulting in increased load overhead; if the fixed handover threshold is large , although the frequency of handover can be reduced, some handovers cannot be triggered in time, resulting in the failure to guarantee the performance of the system.

Contents of the invention

Aiming at the shortcomings of the existing methods, the present disclosure proposes a beam switching method, device, and processor-readable storage medium, so as to solve at least one aspect of the above-mentioned technical defects to a certain extent.

In a first aspect, a beam switching method is provided, performed by a first network node, including:

Acquiring the first quality information respectively corresponding to the multiple first beams in the first time period, the multiple first beams The first quality information respectively corresponding to the beams includes the first quality information sent by at least one user equipment UE to the first network node;

Based on the first quality information respectively corresponding to the multiple first beams, determine the first average channel quality corresponding to the multiple first beams;

Based on the first average channel quality corresponding to a plurality of first beams, and the second average channel quality corresponding to a plurality of second beams in at least one second time period, determine the first state vector; at least one second time period in the first before the time period;

Based on the first state vector, determine a beam switching threshold corresponding to at least one UE.

In an embodiment, obtaining first quality information respectively corresponding to a plurality of first beams within a first time period includes:

Acquire first quality information sent by the second network node within the first time period; and receive first quality information sent by at least one UE;

Wherein, the first quality information includes at least one item of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.

In an embodiment, the first state vector is determined based on the first average channel quality corresponding to the multiple first beams and the second average channel quality corresponding to the multiple second beams in at least one second time period, including:

Performing quantization mapping processing on the first average channel quality to obtain the value of the first information vector corresponding to the first average channel quality; performing quantization mapping processing on the second average channel quality to obtain a second information vector corresponding to the second average channel quality value;

Based on the first information vector and at least one second information vector, a first state vector is obtained.

In an embodiment, based on the first state vector, determining a beam switching threshold corresponding to at least one UE includes:

Input the first state vector into the first relationship model, perform matching processing, obtain one or more reward values matching the first state vector, and set the beam switching threshold corresponding to the maximum reward value among the one or more reward values , determined as the beam switching threshold corresponding to the UE;

Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.

In one embodiment, the first relationship model uses the first index change of at least one UE to The corresponding first parameter and the second parameter corresponding to the beam switching cost are determined.

In one embodiment, the first relational model is trained by:

Constructing a training sample set; based on the training sample set, training the relational model to obtain a first relational model;

Based on the training sample set, the relational model is trained, at least including:

Inputting the second state vector in the training sample set to the relational model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

Based on the loss function value, the model parameters of the relational model are updated to obtain the updated relational model.

In one embodiment, based on the training sample set, the relational model is trained, further comprising:

When the termination condition is not met, the following steps are repeated:

Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the updated relationship model;

Based on the loss function value, the model parameters of the updated relational model are updated to obtain the updated relational model.

In one embodiment, after determining the loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model, it further includes:

Determine whether the termination condition is met;

If it is determined that the termination condition is met, the first relational model is obtained;

The termination condition is one of the following:

The loss function value is less than or equal to the loss function value threshold; or,

The loss function value is greater than or equal to the loss function value threshold.

In an embodiment, the first quality information sent by at least one UE to the first network node is sent to the second network node.

In a second aspect, a beam switching method is provided, which is performed by a UE, including:

Sending the first quality information corresponding to the first beam in the first time period to the first network node; so that the first network node obtains the first quality information respectively corresponding to the multiple beams in the first time period, and based on the multiple beams respectively The corresponding first quality information, and the average channel quality corresponding to multiple beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

Receive the beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

In a third aspect, a beam switching device is provided, which is applied to a first network node, and includes a memory, a transceiver, and a processor:

memory for storing computer programs;

a transceiver, configured to send and receive data under the control of the processor;

A processor that reads a computer program in memory and does the following:

Acquiring first quality information respectively corresponding to multiple first beams within a first time period, where the first quality information respectively corresponding to multiple first beams includes first quality information sent to the first network node by at least one user equipment UE;

Based on the first quality information respectively corresponding to the multiple first beams, determine the first average channel quality corresponding to the multiple first beams;

Based on the first average channel quality corresponding to a plurality of first beams, and the second average channel quality corresponding to a plurality of second beams in at least one second time period, determine the first state vector; at least one second time period in the first before the time period;

Based on the first state vector, determine a beam switching threshold corresponding to at least one UE.

In a fourth aspect, a beam switching device is provided, which is applied to a UE, including a memory, a transceiver, and a processor:

memory for storing computer programs;

a transceiver, configured to send and receive data under the control of the processor;

A processor that reads a computer program in memory and does the following:

Sending the first quality information corresponding to the first beam in the first time period to the first network node; so that the first network node obtains the first quality information respectively corresponding to the multiple beams in the first time period, and based on the multiple beams respectively The corresponding first quality information, and the average channel quality corresponding to multiple beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

Receive the beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

In a fifth aspect, the present disclosure provides a beam switching device applied to a first network node, including:

The first processing unit is configured to acquire first quality information respectively corresponding to multiple first beams within a first time period, where the first quality information respectively corresponding to multiple first beams includes at least one user equipment UE sent to the first network node the first quality information of

The second processing unit is configured to determine the first average channel quality corresponding to the multiple first beams based on the first quality information respectively corresponding to the multiple first beams;

A third processing unit, configured to determine a first state vector based on a first average channel quality corresponding to a plurality of first beams, and a second average channel quality corresponding to a plurality of second beams in at least one second time period; at least one the second time period is before the first time period;

The fourth processing unit is configured to determine a beam switching threshold corresponding to at least one UE based on the first state vector.

In a sixth aspect, the present disclosure provides a beam switching device applied to a UE, including:

The fifth processing unit is configured to send the first quality information corresponding to the first beam within the first time period to the first network node; so that the first network node obtains the first quality information respectively corresponding to the multiple beams within the first time period , and based on the first quality information respectively corresponding to the multiple beams, and the average channel quality corresponding to the multiple beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

The sixth processing unit is configured to receive the beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

In a seventh aspect, a processor-readable storage medium is provided, wherein the processor-readable storage medium stores a computer program, and when the computer program is executed by a processor, the methods described in the first aspect and the second aspect are implemented.

The technical solutions provided by the embodiments of the present disclosure have at least the following beneficial effects:

A dynamic adjustment of the beam switching threshold corresponding to at least one UE is realized, so that an appropriate beam switching frequency can be guaranteed while system performance can be maintained.

Additional aspects and advantages of the disclosure will be set forth in part in the description which follows, and will become apparent from the description, or may be learned by practice of the disclosure.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present disclosure more clearly, the following briefly introduces the drawings that are required for the description of the embodiments of the present disclosure.

FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present disclosure;

FIG. 2 is a schematic flowchart of a beam switching method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of beam switching provided by an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart of another beam switching method provided by an embodiment of the present disclosure;

FIG. 5 is a schematic flow diagram of reinforcement learning provided by an embodiment of the present disclosure;

FIG. 6 is a schematic flowchart of another beam switching method provided by an embodiment of the present disclosure;

FIG. 7 is a schematic flowchart of another beam switching method provided by an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of SINR statistics provided by an embodiment of the present disclosure;

FIG. 9 is a schematic diagram of SINR statistics provided by an embodiment of the present disclosure;

FIG. 10 is a schematic diagram of counting the number of handovers provided by an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of a beam switching device provided by an embodiment of the present disclosure;

FIG. 12 is a schematic structural diagram of a beam switching device provided by an embodiment of the present disclosure;

FIG. 13 is a schematic structural diagram of a beam switching device provided by an embodiment of the present disclosure;

Fig. 14 is a schematic structural diagram of a beam switching device provided by an embodiment of the present disclosure.

Detailed ways

Embodiments of the present application are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present application, and are not construed as limiting the present application.

Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the specification of the present application refers to the presence of said features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Additionally, "connected" or "coupled" as used herein may include wireless connection or wireless coupling. The expression "and/or" used herein includes all or any elements and all combinations of one or more associated listed items.

The term "and/or" in the embodiments of this application describes the association relationship of associated objects, indicating that there may be three relationships, for example, A and/or B, which may mean: A exists alone, A and B exist simultaneously, and B exists alone These three situations. The character "/" generally indicates that the contextual objects are an "or" relationship. The term "plurality" in the embodiments of the present application refers to two or more, and other quantifiers are similar.

"Determine B based on A" in this application means that the factor A should be considered when determining B. It is not limited to "B can be determined only based on A", but also includes: "Determine B based on A and C", "Determine B based on A, C and E", "determine C based on A, further determine B based on C" wait. In addition, it may also include A as the condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; another example, "when A meets the second condition, determine B"; another example , "when A satisfies the third condition, determine B based on the first parameter" and so on. Of course, it may also be a condition that takes A as a factor for determining B, for example, "when A satisfies the first condition, use the first method to determine C, and further determine B based on C" and so on.

"Determine B based on A" in this application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A only", but also includes: "B is determined based on A and C", "B is determined based on A, C and E", "C is determined based on A, and B is further determined based on C" wait. In addition, it may also include A as the condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; another example, "when A meets the second condition, determine B"; another example , "when A satisfies the third condition, determine B based on the first parameter" and so on. Of course, it may also be a condition that takes A as a factor for determining B, for example, "when A satisfies the first condition, use the first method to determine C, and further determine B based on C" and so on.

In order to better understand and illustrate the solutions of the embodiments of the present disclosure, some technical terms involved in the embodiments of the present disclosure will be briefly described below.

(1) Beam switching technology

With the increasing demand for data traffic in communication systems, millimeter wave technology has become a key technology in 5G (5th Generation Mobile Communication Technology, fifth generation mobile communication technology). Due to the high-frequency characteristics of millimeter waves, their links may suffer severe path loss. In order to overcome this shortcoming, millimeter wave communication systems need to use beamforming-based Massive MIMO (Multiple-input Multiple-output, large-scale multiple-input More technology) technology. Beamforming technology can effectively improve the spectral efficiency of mobile communication systems. However, the coverage of each beam is limited, and beam switching is required to maintain good system performance. For example, when the UE leaves the coverage of the serving beam, the serving beam will no longer be suitable for the UE, and beam switching is required, or there is an obstacle between the network node and the UE that causes the quality of the current serving beam to drop suddenly, and beam switching is also required.

In practical application scenarios, network nodes such as millimeter-wave base stations have a smaller coverage area, so the deployment density of millimeter-wave base stations is higher than that of LTE (Long Term Evolution, long-term evolution) base stations. In mmWave networks using beams, beam switching occurs not only between mmWave base stations, but also between beams within the same mmWave base station. The dense deployment of mmWave base stations and the use of beams will further increase the frequency of beam switching, which may limit the performance of practical systems, so the beam switching process needs to be optimized.

In the process of beam management, beam scanning and reporting are performed periodically. Based on the results of beam reporting, when the quality of the current serving beam fluctuates, beam switching can be performed to obtain better communication performance. For this type of beam switching, switching parameters can be optimized to improve system performance and reduce the probability of beam failure, while reducing the number of beam switching times to reduce system load overhead.

(2) Beam switching based on fixed threshold (fixed switching threshold)

For a network node such as a base station, the beam switching based on the measurement report includes two parts of switching: one part is beam switching across base stations, and the other part is beam switching inside the base station. According to actual simulation statistics, most beam switching occurs between different base stations. Therefore refer to cell handover In the A3 event in the mode, after the UE accesses a network node, such as an AP (Access Point, access point), the beam scanning is performed periodically, and after the UE performs beam measurement on all beams, it saves the best N beam sequence numbers and its quality, and obtain candidate beams, the UE feeds back relevant beam information (such as SINR (Signal to Interference plus Noise Ratio, signal to interference and noise ratio), etc.) to the AP, and the AP determines whether the beam should be switched according to the fixed threshold determination method judge.

Use SINR as the reference index of beam quality; report the trigger judgment of handover for all beam qualities each time. The parameters related to the determination method of the fixed threshold include: TTT (Time-to-trigger, trigger time), HM (Hysteresis Margin, hysteresis threshold), etc. HM indicates the minimum difference between the SINR of the current serving beam and the SINR of the target beam, and the threshold is denoted as Δ, and TTT indicates that the target beam satisfies the condition of triggering the difference for a certain period of time. In the fixed-threshold-based handover setting, beam handoff is triggered if the quality of other beams measured to be better than the serving beam exceeds a certain threshold Δ, and this condition persists for a period of time (TTT). The handover judgment process includes: the AP judges whether the candidate beam enters the beam handover judgment process based on the reported beam quality. If the condition of SINR _serve +Δ<SINR _candidate is met, the handover judgment is entered. The judgment process needs to last for TTT time; After the switching judgment process, if the condition of SINR _serve +Δ<SINR _candidate is satisfied within the TTT time, beam switching will occur after the judgment ends, and the serving beam will be switched to the candidate beam; if the above conditions are not satisfied at a certain moment, the judgment will be interrupted process, this judgment ends, and beam switching does not occur.

In the handover decision process, select the beam with the largest SINR in the measurement results of each UE in the beam report as the candidate beam and add it to the candidate beam set; The larger beam is the candidate beam, and it will be pushed forward accordingly. If the candidate beam satisfies the requirement of the threshold, the handover decision process is triggered. During the determination process of beam switching, the candidate beam will not be selected by other users, and at the same time, the user will not select other beams for switching determination until a beam switching determination ends.

The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, not all of them. Based on the embodiments in this disclosure, those of ordinary skill in the art All other embodiments obtained under the premise of making creative work belong to the protection scope of the present disclosure.

A schematic diagram of a system architecture provided by an embodiment of the present disclosure is shown in FIG. 1 . The system architecture includes: UEs and network nodes, wherein UEs are, for example, UE110, UE111, UE112, and UE113 in FIG. 1 , and network nodes are, for example, UEs in FIG. Network node 120 and network node 121. The network nodes are deployed in the access network, for example, the network nodes are deployed in the access network NG-RAN (New Generation-Radio Access Network, New Generation Radio Access Network) in the 5G system. The UE and the network node communicate with each other through a certain air interface technology, for example, they may communicate with each other through a cellular technology.

The UE involved in the embodiments of the present disclosure may be a device that provides voice and/or data connectivity to a user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. Types of UEs include mobile phones, vehicle user terminals, tablets, laptops, personal digital assistants, mobile Internet devices, wearable devices, and the like.

The network node involved in this embodiment of the present disclosure may be a base station, and the base station may include multiple cells that provide services for a terminal. Depending on the specific application, the base station can also be called an access point AP, or it can be a device in the access network that communicates with wireless terminal devices through one or more sectors on the air interface, or other names. The network node is operable to interchange received over-the-air frames with Internet Protocol (IP) packets, acting as a router between the wireless terminal device and the rest of the access network, which may include the Internet Protocol (IP) communication network. The network nodes may also coordinate attribute management for the air interface. For example, the network node involved in the embodiments of the present disclosure may be a network device (Base Transceiver Station, BTS) in Global System for Mobile communications (GSM) or Code Division Multiple Access (Code Division Multiple Access, CDMA) ), it can also be a network device (NodeB) in Wide-band Code Division Multiple Access (WCDMA), or it can be an evolved network device in a long term evolution (long term evolution, LTE) system (evolutional Node B, eNB or e-NodeB), 5G base station (gNB) in the 5G network architecture (next generation system), B5G base station in the super 5th generation mobile communication system (B5G), 6G (6th Generation Mobile Communication Technology, The 6G base station in the network architecture of the sixth generation mobile communication technology can also be a home The home evolved base station (Home evolved Node B, HeNB), relay node (relay node), home base station (femto), pico base station (pico), millimeter wave base station, etc. are not limited in the embodiments of the present disclosure. In some network structures, network nodes may include centralized unit (centralized unit, CU) nodes and distributed unit (distributed unit, DU) nodes, and the centralized unit and distributed unit may also be arranged geographically separately.

In order to make the purpose, technical solution and advantages of the present disclosure clearer, the implementation manners of the present disclosure will be further described in detail below in conjunction with the accompanying drawings.

An embodiment of the present disclosure provides a beam switching method, which is executed by the first network node. The flowchart of the method is shown in FIG. 2 , and the method includes:

S201. Acquire first quality information respectively corresponding to a plurality of first beams within a first time period, where the first quality information respectively corresponding to a plurality of first beams includes first quality information sent by at least one user equipment UE to a first network node .

Specifically, the first network node acquires first quality information respectively corresponding to multiple beams of at least one second network node within the first time period; and receives the first quality information sent by at least one UE. Wherein, the first quality information may include at least one of SINR, RSRP (Reference Signal Receiving Power, received power), and RSRQ (Reference Signal Receiving Quality, received quality). For example, the first network node obtains first quality information corresponding to 20 first beams in the first time period, that is, in the first time period, the first network node obtains 20 first quality information, and the 20 first beams The piece of quality information includes 10 pieces of first quality information sent by one or more second network nodes to the first network node, and 10 pieces of first quality information sent by the 10 UEs to the first network node respectively.

S202. Based on the first quality information respectively corresponding to the multiple first beams, determine first average channel quality corresponding to the multiple first beams.

Specifically, for example, the first network node obtains 20 pieces of first quality information, the first quality information is SINR, and the 20 pieces of first quality information are 20 SINRs, then the average value of these 20 SINRs is calculated, and the The average value of 20 SINRs is used as the first average channel quality.

S203. Determine a first state vector based on the first average channel quality corresponding to multiple first beams and the second average channel quality corresponding to multiple second beams in at least one second time period; at least one second time period is in Before the first time period.

Specifically, as shown in Figure 3, the sliding window includes N time windows, and these N time windows are respectively the first time window, the second time window, the third time window, ... the Nth time window, the Nth time window The time lengths of one time window, the second time window, the third time window, ... the Nth time window can be preset, and N is a positive integer. The first time period is the first time window, and the first time window is the current window; multiple second time periods can be respectively the second time window, the third time window, ... the Nth time window, wherein, The second time window, the third time window, ... the Nth time window are all historical windows; the average channel quality of the first time window can be the first average channel quality, and the average channel quality of the second time window can be the first The average channel quality of the second time window, the average channel quality of the third time window can be the second average channel quality, ... The average channel quality of the Nth time window can be the second average channel quality; the average channel quality of the second time window, the third The average channel quality of the time window, ... the average channel quality of the Nth time window can be different; the average channel quality of the first time window can be expressed by SINR ₁ , the average channel quality of the second time window, and the average channel quality of the third time window The quality, ... the average channel quality of the Nth time window can be represented by SINR ₂ , SINR ₃ , ... SINR _N respectively.

It should be noted that, taking into account the changing trend of system performance over time and avoiding unnecessary beam switching in the future, the use of sliding windows can take into account the influence of historical experience.

S204. Based on the first state vector, determine a beam switching threshold corresponding to at least one UE.

Specifically, the beam switching threshold may include TTT, HM, and the like.

In the embodiments of the present disclosure, it is realized that the beam switching threshold corresponding to at least one UE is dynamically adjusted, so that while maintaining system performance, an appropriate beam switching frequency can also be guaranteed.

In an embodiment, obtaining first quality information respectively corresponding to a plurality of first beams within a first time period includes:

Acquire first quality information sent by the second network node within the first time period; and receive first quality information sent by at least one UE;

Wherein, the first quality information includes at least one item of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.

In one embodiment, the first network node can obtain at least one second The multiple beams of the network node respectively correspond to the first quality information; and receive the first quality information sent by at least one UE.

In an embodiment, the first state vector is determined based on the first average channel quality corresponding to the multiple first beams and the second average channel quality corresponding to the multiple second beams in at least one second time period, including:

Performing quantization mapping processing on the first average channel quality to obtain the value of the first information vector corresponding to the first average channel quality; performing quantization mapping processing on the second average channel quality to obtain a second information vector corresponding to the second average channel quality value;

Based on the first information vector and at least one second information vector, a first state vector is obtained.

Specifically, the value of the first information vector and the value of the second information vector are both discrete values, the value of the first information vector and the value of the second information vector can be used to represent the signal strength level, and the signal strength level can be 1 or 2 , 3, ... m, etc., wherein, m is a positive integer. For another example, the first information vector is b ₁ , multiple second information vectors can be b ₂ , b ₃ , ... b _n respectively, then the first state vector is (b ₁ , b ₂ , b ₃ , ... b _n ) .

In an embodiment, based on the first state vector, determining a beam switching threshold corresponding to at least one UE includes:

Input the first state vector into the first relationship model, perform matching processing, obtain one or more reward values matching the first state vector, and set the beam switching threshold corresponding to the maximum reward value among the one or more reward values , determined as the beam switching threshold corresponding to the UE;

Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.

Specifically, the first relationship model may be a table, and the table may be used to characterize the relationship among the first state vector, the reward value, and the beam switching threshold. The reward value may be used to represent the effect of the UE performing beam switching based on the UE's corresponding beam switching threshold. The maximum reward value can be used to represent the best effect of the UE performing beam switching based on the UE's corresponding beam switching threshold.

In an embodiment, the first relationship model is determined by a first parameter corresponding to a first index change of at least one UE, and a second parameter corresponding to a beam switching cost.

Specifically, the first index may include at least one of channel capacity, signal-to-noise ratio, and block error rate BLER.

In an embodiment, the first relational model may be a table, and the table may be determined by using a first parameter corresponding to a channel capacity change of at least one UE and a second parameter corresponding to a beam switching cost.

Specifically, the first parameter corresponding to the channel capacity change of at least one UE may be The second parameter corresponding to the beam switching cost may be β _c *H _n , where, Indicates that the beam switching threshold corresponding to the state vector results in change in capacity, represents the average value of channel capacity C of at least one UE, β _c is a preset penalty factor, and H _n is an average number of handovers.

In one embodiment, the first relational model can be trained by reinforcement learning. For example, the first relational model is trained by:

Constructing a training sample set; based on the training sample set, training the relational model to obtain a first relational model;

Based on the training sample set, the relational model is trained, at least including:

Inputting the second state vector in the training sample set to the relational model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

Based on the loss function value, the model parameters of the relational model are updated to obtain the updated relational model.

In one embodiment, after determining the loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model, it further includes:

Determine whether the termination condition is met;

If it is determined that the termination condition is met, the first relational model is obtained;

The termination condition is one of the following:

The loss function value is less than or equal to the loss function value threshold; or,

The loss function value is greater than or equal to the loss function value threshold.

Specifically, the second state vector in the training sample set is input to the relational model, and the beam switching threshold corresponding to the second state vector is determined; based on the beam switching threshold corresponding to the second state vector, Through the reward function, the reward value corresponding to the second state vector is obtained; the second state vector and the reward value corresponding to the second state vector are substituted into the loss function corresponding to the relational model to obtain the loss function value, and based on the loss function value, the relational model Model parameters are updated. Until the termination condition is reached, the relational model when the termination condition is reached is taken as the first relational model.

In one embodiment, based on the training sample set, the relational model is trained, further comprising:

When the termination condition is not met, the following steps are repeated:

Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the updated relationship model;

Based on the loss function value, the model parameters of the updated relational model are updated to obtain the updated relational model.

Specifically, if the first index is channel capacity, input the second state vector in the training sample set to the relational model, and determine the beam switching threshold corresponding to the second state vector based on the greedy strategy; the second state vector is used to represent each training Average channel quality across multiple beams over time period.

Based on the beam switching threshold corresponding to the second state vector, the reward value corresponding to the second state vector is obtained through the reward function; wherein, the reward function is shown in formula (1):

in, Indicates that the beam switching threshold corresponding to the second state vector results in change in capacity, represents the average value of channel capacity C of at least one UE, β _c is a preset penalty factor, and H _n is an average number of handovers.

The first relational model can be deployed in a network node.

In an embodiment, the first quality information sent by at least one UE to the first network node is sent to the second network node.

Specifically, the first network node may share the first quality information as side information with other network nodes (for example, one or more second network nodes); wherein, the side information may be Information shared between network nodes.

An embodiment of the present disclosure provides another beam switching method, which is performed by the UE. The flowchart of the method is shown in FIG. 4 , and the method includes:

S401. Send the first quality information corresponding to the first beam in the first time period to the first network node; so that the first network node obtains the first quality information corresponding to the multiple beams in the first time period, and based on multiple The first quality information corresponding to the beams and the average channel quality corresponding to the multiple beams in at least one second time period determine the beam switching threshold corresponding to the UE.

S402. Receive a beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

It should be noted that if the UE measures that the quality of other beams other than the serving beam is better than that of the serving beam (such as the first beam), beam switching may be triggered based on the beam switching threshold, so that the UE switches from the serving beam to other beams. beam.

In the embodiments of the present disclosure, it is realized that the beam switching threshold corresponding to the UE is dynamically adjusted, so that while maintaining system performance, an appropriate beam switching frequency can also be ensured.

The beam switching method in the above-mentioned embodiments of the present disclosure is introduced comprehensively and in detail through the following embodiments:

In the embodiments of the present disclosure, reinforcement learning in the beam switching scenario is provided. The flow diagram of the reinforcement learning is shown in FIG. 5 , including:

S501. Preset a beam switching threshold.

Specifically, the beam switching threshold may include TTT, HM, and the like.

S502. Construct a training sample set.

Specifically, each state vector in the training sample set can be used to represent the average channel quality of multiple beams within a training period.

S503, preset a reward function.

Specifically, a reward function as shown in formula (1) is set.

S504. Based on the training sample set, train the relationship model to obtain the trained relationship model.

Specifically, the trained relational model is the first relational model. According to the Q-learning algorithm, based on the training sample set, the relational model is trained. The relational model can be a table, and the table can be a Q-table (Q table) in the Q-learning algorithm, and the table can be used to represent the state vector, award The relationship between the excitation value and the beam switching threshold; input the state vector in the training sample set to the table, and determine the beam switching threshold corresponding to the state vector; based on the beam switching threshold corresponding to the state vector, through the reward function, get the corresponding Reward value; Substitute the state vector and the reward value corresponding to the state vector into the loss function corresponding to the table to obtain the loss function value, and update the model parameters of the table based on the loss function value. Repeat the following steps: input the state vector in the training sample set to the table, and determine the beam switching threshold corresponding to the state vector; based on the beam switching threshold corresponding to the state vector, obtain the reward value corresponding to the state vector through the reward function; The reward value corresponding to the state vector is substituted into the loss function corresponding to the table to obtain the loss function value, and based on the loss function value, the model parameters of the table are updated. Until the termination condition is reached, the table when the termination condition is reached is used as the first table (relational model after training); wherein, the termination condition can be that the loss function value is less than or equal to the loss function value threshold.

S505. Deploy the trained relationship model on the network nodes.

Specifically, the trained relationship model may be the first table, and the first table is deployed in the network nodes.

Another beam switching method is provided in the embodiment of the present disclosure. The schematic flowchart of the method is shown in FIG. 6 , and the method includes:

S601, the network node sends a reference signal for beam scanning to the UE.

Specifically, the reference signal may be an SSB (Synchronization Signal Block, synchronization signal block).

S602. The UE performs beam measurement to obtain quality information corresponding to the serving beam.

S603, the UE feeds back the quality information corresponding to the serving beam and the quality information corresponding to the candidate beam to the network node.

S604. The network node uses the quality information corresponding to the serving beam as side information, and shares the side information with other network nodes.

It should be noted that, if there are no other network nodes, step S604 is not required; the quality information corresponding to the serving beam may be SINR, RSRP or RSRQ.

S605. The network node determines a state vector based on all side information, and searches for a beam switching threshold corresponding to the UE based on a relationship model obtained through reinforcement learning training.

Specifically, the network node stores all edge information, and all edge information is the edge corresponding to the serving beam information, and side information sent by other network nodes to network nodes.

S606, the network node makes a beam switching decision based on the beam switching threshold; if the network node determines that the UE switches from the serving beam to other beams, the network node instructs the UE to switch from the serving beam to other beams, and proceeds to step S607 for execution.

Specifically, other beams may be candidate beams.

S607, the UE performs beam switching.

The embodiment of the present disclosure provides yet another beam switching method, which is applied in a multi-AP millimeter wave network scenario, and the topology of the scenario is: in a scenario with a radius of 30m, 3 APs and 7 users are evenly deployed; Among them, 3 of the 7 users move along a certain path at a speed of 5 km/h (kilometer/hour), and each of the 3 APs has 32 beams, using hybrid beamforming technology, The cycle of beam scanning is set to 10 ms, and the UE performs beam switching when it meets the conditions for beam switching. A schematic flow chart of the method is shown in Figure 7, and the method includes:

S701. The network node determines a beam switching threshold through a greedy strategy.

Specifically, the options for the beam switching threshold: (HM=1dB and TTT=100ms), (HM=1dB and TTT=100ms), (HM=3dB and TTT=100ms), (HM=5dB and TTT=100ms), (HM=7dB and TTT=100ms) and so on.

S702. The UE performs beam switching based on the beam switching threshold.

S703. The network node determines the second state vector through the sliding window.

For example, the sliding window includes 5 time windows, that is, the length of the sliding window is 5, and these 5 time windows are respectively the first time window, the second time window, the third time window, the fourth time window and For the fifth time window, the time length of the first time window is 10ms, and the time lengths of the second time window, the third time window, the fourth time window and the fifth time window are all 40ms.

S704, the network node updates the Q table through the reward function.

Specifically, the reinforcement learning training is performed according to the reward function shown in formula (1), that is, the training of the relationship model is performed, and the relationship model is a Q table. For example, the penalty factor β _c in the reward function can be set to 0.75.

S705, the network node deploys the updated Q table to the network node; respectively go to step S701 and S706 are executed.

S706. The network node determines the first state vector, and obtains the beam switching threshold by looking up the Q table.

Specifically, the first state vector may be (b ₁ , b ₂ , b ₃ , . . . b _n ). The values of b ₁ , b ₂ , b ₃ ,...b _n can be used to characterize the signal strength level, and the signal strength level can be 1, 2, 3,...m, etc., for example, the signal strength level can be 1, 2, 3 , 4 and 5, that is, there are 5 levels of signal strength.

S707, the network node performs a beam switching decision.

S708. The network node performs periodic beam scanning.

S709, the UE performs beam measurement.

S710, the UE feeds back the quality information corresponding to the beam; go to step S706 for execution.

Specifically, the quality information may be SINR, RSRP or RSRQ.

It should be noted that steps S701-S705 are steps in reinforcement learning training, and the update interval T of each reinforcement learning training may be 10 ms (milliseconds); steps S706-S710 are steps in online execution.

In one embodiment, as shown in Figure 8 and Figure 9 (Figure 9 is an enlarged view of Figure 8), based on a fixed threshold, the CDF (Cumulative Distribution Function, Cumulative Distribution Function) of the SINR corresponding to the serving beam within 20s (seconds) function) statistical results; this disclosure is based on the beam switching method of reinforcement learning (such as Q-learning), and the CDF statistical results of the SINR corresponding to the serving beam within 20s; the performance of the beam switching based on reinforcement learning in this disclosure is better.

In one embodiment, as shown in FIG. 10 , the present disclosure is based on reinforcement learning (such as Q-learning) beam switching, and the number of beam switching occurrences is the least.

Based on the same inventive concept, the embodiment of the present disclosure also provides a beam switching device, which is applied to the first network node. The structural diagram of the device is shown in FIG. Receive and send data.

Wherein, in FIG. 11 , the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by the processor 1310 and various circuits of the memory represented by the memory 1320 are linked together. The bus architecture can also link together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art, Therefore, it is not described further herein. The bus interface provides the interface. The transceiver 1300 may be a plurality of elements, including a transmitter and a receiver, providing a unit for communicating with various other devices over transmission media, including wireless channels, wired channels, optical cables, and other transmission media. The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 is used to store computer programs, such as data used by the processor 1310 when performing operations.

The processor 1310 may be a central processing device (CPU), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or a complex programmable logic device (Complex Programmable Logic Device , CPLD), the processor can also adopt a multi-core architecture.

Processor 1310, configured to read the computer program in the memory and perform the following operations:

Acquiring first quality information respectively corresponding to multiple first beams within a first time period, where the first quality information respectively corresponding to multiple first beams includes first quality information sent to the first network node by at least one user equipment UE;

Based on the first quality information respectively corresponding to the multiple first beams, determine the first average channel quality corresponding to the multiple first beams;

Based on the first average channel quality corresponding to a plurality of first beams, and the second average channel quality corresponding to a plurality of second beams in at least one second time period, determine the first state vector; at least one second time period in the first before the time period;

Based on the first state vector, determine a beam switching threshold corresponding to at least one UE.

In an embodiment, obtaining first quality information respectively corresponding to a plurality of first beams within a first time period includes:

Acquire first quality information sent by the second network node within the first time period; and receive first quality information sent by at least one UE;

Wherein, the first quality information includes at least one item of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.

In an embodiment, the first state vector is determined based on the first average channel quality corresponding to the multiple first beams and the second average channel quality corresponding to the multiple second beams in at least one second time period, including:

The first average channel quality is quantized and mapped to obtain the first average channel quality corresponding to The value of the first information vector; and the second average channel quality is quantized and mapped to obtain the value of the second information vector corresponding to the second average channel quality;

Based on the first information vector and at least one second information vector, a first state vector is obtained.

In an embodiment, based on the first state vector, determining a beam switching threshold corresponding to at least one UE includes:

Input the first state vector into the first relationship model, perform matching processing, obtain one or more reward values matching the first state vector, and set the beam switching threshold corresponding to the maximum reward value among the one or more reward values , determined as the beam switching threshold corresponding to the UE;

Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.

In an embodiment, the first relationship model is determined by a first parameter corresponding to a first index change of at least one UE, and a second parameter corresponding to a beam switching cost.

In one embodiment, the first relational model is trained by:

Constructing a training sample set; based on the training sample set, training the relational model to obtain a first relational model;

Based on the training sample set, the relational model is trained, at least including:

Inputting the second state vector in the training sample set to the relational model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

Based on the loss function value, the model parameters of the relational model are updated to obtain the updated relational model.

In one embodiment, based on the training sample set, the relational model is trained, further comprising:

When the termination condition is not met, the following steps are repeated:

Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

Based on the beam switching threshold and reward function corresponding to the second state vector, determine the direction of the second state to The reward value corresponding to the amount;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the updated relationship model;

Based on the loss function value, the model parameters of the updated relational model are updated to obtain the updated relational model.

In one embodiment, after determining the loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model, it further includes:

Determine whether the termination condition is met;

If it is determined that the termination condition is met, the first relational model is obtained;

The termination condition is one of the following:

The loss function value is less than or equal to the loss function value threshold; or,

The loss function value is greater than or equal to the loss function value threshold.

In an embodiment, the first quality information sent by at least one UE to the first network node is sent to the second network node.

It should be noted here that the above-mentioned device provided by the embodiment of the present invention can realize all the method steps realized by the above-mentioned method embodiment, and can achieve the same technical effect. The part and the beneficial effect are described in detail.

Based on the same inventive concept, an embodiment of the present disclosure also provides a beam switching device, which is applied to a UE. The structural diagram of the device is shown in FIG. data.

Wherein, in FIG. 12 , the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by the processor 1410 and various circuits of the memory represented by the memory 1420 are linked together. The bus architecture can also link together various other circuits such as peripherals, voltage regulators, and power management circuits, etc., which are well known in the art and therefore will not be further described herein. The bus interface provides the interface. Transceiver 1400 may be a plurality of elements, including transmitters and receivers, providing means for communicating with various other devices over transmission media, including wireless channels, wired channels, fiber optic cables, etc. Transmission medium. For different user equipments, the user interface 1430 can also be an interface capable of connecting externally and internally to required devices, and the connected devices include but are not limited to keypads, display screens, etc. monitors, speakers, microphones, joysticks, etc.

The processor 1410 is responsible for managing the bus architecture and general processing, and the memory 1420 can store data used by the processor 1410 when performing operations.

Optionally, the processor 1410 can be a CPU (central device), ASIC (Application Specific Integrated Circuit, application specific integrated circuit), FPGA (Field-Programmable Gate Array, field programmable gate array) or CPLD (Complex Programmable Logic Device , complex programmable logic device), the processor can also adopt a multi-core architecture.

The processor is used to execute the method of the second aspect provided by the embodiments of the present disclosure according to the obtained executable instruction by calling the computer program stored in the memory. The processor and memory may also be physically separated.

Processor 1410, configured to read the computer program in memory 1420 and perform the following operations:

Sending the first quality information corresponding to the first beam in the first time period to the first network node; so that the first network node obtains the first quality information respectively corresponding to the multiple beams in the first time period, and based on the multiple beams respectively The corresponding first quality information, and the average channel quality corresponding to multiple beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

Receive the beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

It should be noted here that the above-mentioned device provided by the embodiment of the present invention can realize all the method steps realized by the above-mentioned method embodiment, and can achieve the same technical effect. The part and the beneficial effect are described in detail.

Based on the same inventive concept as the foregoing embodiments, the embodiments of the present disclosure also provide a beam switching device, which is applied to the first network node. The structural diagram of the device is shown in FIG. 13 . The beam switching device 80 includes a first processing unit 801. A second processing unit 802, a third processing unit 803, and a fourth processing unit 804.

The first processing unit 801 is configured to acquire first quality information corresponding to multiple first beams respectively within a first time period, where the first quality information corresponding to multiple first beams respectively includes at least one user equipment UE sent to the first network The first quality information of the node;

The second processing unit 802 is configured to determine the first average channel quality corresponding to the multiple first beams based on the first quality information respectively corresponding to the multiple first beams;

The third processing unit 803 is configured to determine the first state vector based on the first average channel quality corresponding to the multiple first beams and the second average channel quality corresponding to the multiple second beams in at least one second time period; at least a second time period preceding the first time period;

The fourth processing unit 804 is configured to determine a beam switching threshold corresponding to at least one UE based on the first state vector.

In one embodiment, the first processing unit 801 is specifically configured to:

Acquire first quality information sent by the second network node within the first time period; and receive first quality information sent by at least one UE;

Wherein, the first quality information includes at least one item of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.

In one embodiment, the third processing unit 803 is specifically configured to:

Performing quantization mapping processing on the first average channel quality to obtain the value of the first information vector corresponding to the first average channel quality; performing quantization mapping processing on the second average channel quality to obtain a second information vector corresponding to the second average channel quality value;

Based on the first information vector and at least one second information vector, a first state vector is obtained.

In one embodiment, the fourth processing unit 804 is specifically configured to:

Input the first state vector into the first relationship model, perform matching processing, obtain one or more reward values matching the first state vector, and set the beam switching threshold corresponding to the maximum reward value among the one or more reward values , determined as the beam switching threshold corresponding to the UE;

Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.

In an embodiment, the first relationship model is determined by a first parameter corresponding to a first index change of at least one UE, and a second parameter corresponding to a beam switching cost.

In one embodiment, the first relational model is trained by:

Constructing a training sample set; based on the training sample set, training the relational model to obtain a first relational model;

Based on the training sample set, the relational model is trained, at least including:

Inputting the second state vector in the training sample set to the relational model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

Based on the loss function value, the model parameters of the relational model are updated to obtain the updated relational model.

In one embodiment, based on the training sample set, the relational model is trained, further comprising:

When the termination condition is not met, the following steps are repeated:

Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

Determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

Determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the updated relationship model;

Based on the loss function value, the model parameters of the updated relational model are updated to obtain the updated relational model.

In one embodiment, after determining the loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model, it further includes:

Determine whether the termination condition is met;

If it is determined that the termination condition is met, the first relational model is obtained;

The termination condition is one of the following:

The loss function value is less than or equal to the loss function value threshold; or,

The loss function value is greater than or equal to the loss function value threshold.

In one embodiment, the first processing unit 801 is further configured to:

The first quality information sent by at least one UE to the first network node is sent to the second network node.

It should be noted here that the above-mentioned device provided by the embodiment of the present invention can realize all the method steps realized by the above-mentioned method embodiment, and can achieve the same technical effect. The part and the beneficial effect are described in detail.

Based on the same inventive concept as the foregoing embodiments, the embodiments of the present disclosure also provide a beam switching device, which is applied to a UE. The structural diagram of the device is shown in FIG. Six processing units 902 .

The fifth processing unit 901 is configured to send the first quality information corresponding to the first beam within the first time period to the first network node; so that the first network node obtains the first quality information corresponding to the multiple beams within the first time period information, and based on the first quality information respectively corresponding to the multiple beams, and the average channel quality corresponding to the multiple beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

The sixth processing unit 902 is configured to receive the beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.

It should be noted here that the above-mentioned device provided by the embodiment of the present invention can realize all the method steps realized by the above-mentioned method embodiment, and can achieve the same technical effect. The part and the beneficial effect are described in detail.

It should be noted that the division of units in the embodiment of the present application is schematic, and is only a logical function division, and there may be another division manner in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Based on the same inventive concept, the embodiment of the present disclosure also provides a processor-readable storage medium The essence is to store a computer program, and the computer program is used to implement the steps of any method for determining the time of cell handover provided by any one of the embodiments of the present disclosure or any one of the optional implementations when executed by a processor.

The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (e.g., floppy disk, hard disk, tape, magneto-optical disk (MO), etc.), optical storage (e.g., CD, DVD, BD, HVD, etc.), and semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowcharts and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each procedure and/or block in the flowchart and/or block diagrams, and combinations of procedures and/or blocks in the flowchart and/or block diagrams can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine, such that instructions executed by the processor of the computer or other programmable data processing equipment produce Means for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These processor-executable instructions may also be stored in a processor-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the processor-readable memory produce a manufacturing product, the instruction device realizes the functions specified in one or more procedures of the flow chart and/or one or more blocks of the block diagram.

These processor-executable instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented Execution instructions are provided for implementing A step of a function specified in a flow chart process or processes and/or a block diagram block or blocks.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims (31)

1. A beam switching method, performed by a first network node, including:

   Acquire first quality information respectively corresponding to multiple first beams within a first time period, where the first quality information respectively corresponding to multiple first beams includes the first quality information sent by at least one user equipment UE to the first network node. quality information;

   Based on the first quality information respectively corresponding to the multiple first beams, determine a first average channel quality corresponding to the multiple first beams;

   Based on the first average channel quality corresponding to the plurality of first beams and the second average channel quality corresponding to the plurality of second beams in at least one second time period, determine the first state vector; the at least one second time period period is before said first time period;

   Based on the first state vector, determine a beam switching threshold corresponding to the at least one UE.
2. The method according to claim 1, wherein said obtaining first quality information respectively corresponding to a plurality of first beams within a first time period comprises:

   Acquiring the first quality information sent by the second network node within the first time period; and receiving the first quality information sent by the at least one UE;

   Wherein, the first quality information includes at least one of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.
3. The method according to claim 1, wherein, based on the first average channel quality corresponding to the plurality of first beams and the second average channel quality corresponding to a plurality of second beams in at least one second time period, Determine the first state vector, including:

   performing quantization and mapping processing on the first average channel quality to obtain the value of the first information vector corresponding to the first average channel quality; and performing quantization mapping processing on the second average channel quality to obtain the second average The value of the second information vector corresponding to the channel quality;

   A first state vector is obtained based on the first information vector and at least one of the second information vectors.
4. The method according to claim 1, wherein the determining the beam switching threshold corresponding to the at least one UE based on the first state vector comprises:

   Input the first state vector into the first relational model, perform matching processing, and obtain the one or more reward values that match the first state vector, and determine a beam switching threshold corresponding to a maximum reward value among the one or more reward values as the beam switching threshold corresponding to the UE;

   Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.
5. The method according to claim 4, wherein the first relationship model is determined by a first parameter corresponding to a first index change of the at least one UE and a second parameter corresponding to a beam switching cost.
6. The method according to claim 4, wherein the first relational model is obtained through training in the following manner:

   Constructing a training sample set; based on the training sample set, training the relational model to obtain the first relational model;

   The training of the relational model based on the training sample set includes at least:

   inputting a second state vector in the training sample set to the relationship model, and determining a beam switching threshold corresponding to the second state vector;

   determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

   determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

   Based on the loss function value, the model parameters of the relationship model are updated to obtain an updated relationship model.
7. The method according to claim 6, wherein said training a relational model based on said training sample set further comprises:

   When the termination condition is not met, the following steps are repeated:

   Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

   determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

   Based on the second state vector, the reward value corresponding to the second state vector and the updated The loss function corresponding to the relational model, and determine the value of the loss function;

   Based on the loss function value, the model parameters of the updated relationship model are updated to obtain the updated relationship model.
8. The method according to claim 6, wherein, after determining the loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model, further comprising:

   Determine whether the termination condition is met;

   If it is determined that the termination condition is met, then obtain the first relational model;

   The termination condition is one of the following:

   The loss function value is less than or equal to a loss function value threshold; or,

   The loss function value is greater than or equal to a loss function value threshold.
9. The method according to claim 1, further comprising:

   Sending the first quality information sent by the at least one UE to the first network node to a second network node.
10. A beam switching method, performed by a UE, including:

    Sending the first quality information corresponding to the first beam within the first time period to the first network node; so that the first network node obtains the first quality information respectively corresponding to the multiple beams within the first time period, and based on The first quality information corresponding to the plurality of beams, and the average channel quality corresponding to the plurality of beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

    receiving the beam switching threshold corresponding to the UE sent by the first network node, and performing beam switching based on the beam switching threshold.
11. A beam switching device, applied to a first network node, includes a memory, a transceiver, and a processor, wherein:

    memory for storing computer programs;

    a transceiver, configured to send and receive data under the control of the processor;

    a processor for reading the computer program in said memory and performing the following operations:

    Acquire first quality information respectively corresponding to multiple first beams within a first time period, where the first quality information respectively corresponding to multiple first beams includes at least one user equipment UE sent to the first quality information of the first network node;

    Based on the first quality information respectively corresponding to the multiple first beams, determine a first average channel quality corresponding to the multiple first beams;

    Based on the first average channel quality corresponding to the plurality of first beams and the second average channel quality corresponding to the plurality of second beams in at least one second time period, determine the first state vector; the at least one second time period period is before said first time period;

    Based on the first state vector, determine a beam switching threshold corresponding to the at least one UE.
12. The device according to claim 11, wherein said acquiring first quality information respectively corresponding to a plurality of first beams within a first time period comprises:

    Acquiring the first quality information sent by the second network node within the first time period; and receiving the first quality information sent by the at least one UE;

    Wherein, the first quality information includes at least one of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.
13. The apparatus according to claim 11, wherein, based on the first average channel quality corresponding to the multiple first beams and the second average channel quality corresponding to multiple second beams in at least one second time period, Determine the first state vector, including:

    performing quantization and mapping processing on the first average channel quality to obtain the value of the first information vector corresponding to the first average channel quality; and performing quantization mapping processing on the second average channel quality to obtain the second average The value of the second information vector corresponding to the channel quality;

    A first state vector is obtained based on the first information vector and at least one of the second information vectors.
14. The apparatus according to claim 11, wherein the determining the beam switching threshold corresponding to the at least one UE based on the first state vector comprises:

    Inputting the first state vector into the first relational model, performing matching processing, obtaining one or more reward values matching the first state vector, and rewarding the largest reward among the one or more reward values The beam switching threshold corresponding to the value is determined as the beam switching threshold corresponding to the UE;

    Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.
15. The apparatus according to claim 14, wherein the first relationship model is determined by a first parameter corresponding to a first index change of the at least one UE and a second parameter corresponding to a beam switching cost.
16. The device according to claim 14, wherein the first relationship model is obtained by training in the following manner:

    Constructing a training sample set; based on the training sample set, training the relational model to obtain the first relational model;

    The training of the relational model based on the training sample set includes at least:

    inputting a second state vector in the training sample set to the relationship model, and determining a beam switching threshold corresponding to the second state vector;

    determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

    determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

    Based on the loss function value, the model parameters of the relationship model are updated to obtain an updated relationship model.
17. The device according to claim 16, wherein said training a relational model based on said training sample set further comprises:

    When the termination condition is not met, the following steps are repeated:

    Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

    determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

    determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the updated relationship model;

    Based on the loss function value, the model parameters of the updated relationship model are updated to obtain the updated relationship model.
18. The apparatus according to claim 16, wherein, in said second state vector based, The reward value corresponding to the second state vector and the loss function corresponding to the relationship model, after determining the loss function value, further include:

    Determine whether the termination condition is met;

    If it is determined that the termination condition is met, then obtain the first relational model;

    The termination condition is one of the following:

    The loss function value is less than or equal to a loss function value threshold; or,

    The loss function value is greater than or equal to a loss function value threshold.
19. The apparatus according to claim 11, wherein the operations performed by the processor to read the computer program in the memory further include:

    Sending the first quality information sent by the at least one UE to the first network node to the second network node.
20. A beam switching device, applied to a UE, including a memory, a transceiver, and a processor, wherein:

    memory for storing computer programs;

    a transceiver, configured to send and receive data under the control of the processor;

    a processor for reading the computer program in said memory and performing the following operations:

    Sending the first quality information corresponding to the first beam within the first time period to the first network node; so that the first network node obtains the first quality information respectively corresponding to the multiple beams within the first time period, and based on The first quality information corresponding to the plurality of beams, and the average channel quality corresponding to the plurality of beams in at least one second time period, determine the beam switching threshold corresponding to the UE;

    receiving the beam switching threshold corresponding to the UE sent by the first network node, and performing beam switching based on the beam switching threshold.
21. A beam switching device, applied to a first network node, comprising:

    The first processing unit is configured to obtain first quality information respectively corresponding to multiple first beams within a first time period, where the first quality information respectively corresponding to multiple first beams includes at least one user equipment UE sent to the first quality information of the first network node;

    A second processing unit, configured to determine a first average channel quality corresponding to the plurality of first beams based on the first quality information respectively corresponding to the plurality of first beams;

    A third processing unit, configured to determine a first state vector based on a first average channel quality corresponding to the plurality of first beams and a second average channel quality corresponding to a plurality of second beams in at least one second time period; said at least one second time period precedes said first time period;

    A fourth processing unit, configured to determine a beam switching threshold corresponding to the at least one UE based on the first state vector.
22. The device according to claim 21, wherein the first processing unit acquires first quality information respectively corresponding to a plurality of first beams within a first time period, comprising:

    Acquiring the first quality information sent by the second network node within the first time period; and receiving the first quality information sent by the at least one UE;

    Wherein, the first quality information includes at least one of signal to interference and noise ratio SINR, received power RSRP, and received quality RSRQ.
23. The apparatus according to claim 21, wherein the third processing unit is based on the first average channel quality corresponding to the plurality of first beams and the second channel quality corresponding to the plurality of second beams in at least one second time period. Average channel quality, determine the first state vector, comprising:

    performing quantization and mapping processing on the first average channel quality to obtain the value of the first information vector corresponding to the first average channel quality; and performing quantization mapping processing on the second average channel quality to obtain the second average The value of the second information vector corresponding to the channel quality;

    A first state vector is obtained based on the first information vector and at least one of the second information vectors.
24. The apparatus according to claim 21, wherein the fourth processing unit determines the beam switching threshold corresponding to the at least one UE based on the first state vector, comprising:

    Inputting the first state vector into the first relational model, performing matching processing, obtaining one or more reward values matching the first state vector, and rewarding the largest reward among the one or more reward values The beam switching threshold corresponding to the value is determined as the beam switching threshold corresponding to the UE;

    Wherein, the first relationship model is used to characterize the relationship among the first state vector, reward value and beam switching threshold.
25. The apparatus according to claim 24, wherein the first relationship model changes the first parameter corresponding to the first indicator of the at least one UE, and the beam switching cost corresponds to The second parameter of is determined.
26. The device according to claim 24, wherein the first relational model is trained by:

    Constructing a training sample set; based on the training sample set, training the relational model to obtain the first relational model;

    The training of the relational model based on the training sample set includes at least:

    inputting a second state vector in the training sample set to the relationship model, and determining a beam switching threshold corresponding to the second state vector;

    determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

    determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model;

    Based on the loss function value, the model parameters of the relationship model are updated to obtain an updated relationship model.
27. The device according to claim 26, wherein said training a relational model based on said training sample set further comprises:

    When the termination condition is not met, the following steps are repeated:

    Inputting the second state vector in the training sample set to the updated relationship model, and determining the beam switching threshold corresponding to the second state vector;

    determining a reward value corresponding to the second state vector based on the beam switching threshold and the reward function corresponding to the second state vector;

    determining a loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the updated relationship model;

    Based on the loss function value, the model parameters of the updated relationship model are updated to obtain the updated relationship model.
28. The device according to claim 26, wherein, after determining the loss function value based on the second state vector, the reward value corresponding to the second state vector, and the loss function corresponding to the relationship model, further comprising:

    Determine whether the termination condition is met;

    If it is determined that the termination condition is met, then obtain the first relational model;

    The termination condition is one of the following:

    The loss function value is less than or equal to a loss function value threshold; or,

    The loss function value is greater than or equal to a loss function value threshold.
29. The device according to claim 21, wherein the first processing unit is further configured to:

    Sending the first quality information sent by the at least one UE to the first network node to the second network node.
30. A beam switching device, applied to a UE, comprising:

    The fifth processing unit is configured to send the first quality information corresponding to the first beam within the first time period to the first network node; so that the first network node obtains the quality information respectively corresponding to the multiple beams within the first time period First quality information, and based on the first quality information respectively corresponding to the plurality of beams, and the average channel quality corresponding to the plurality of beams in at least one second time period, determine a beam switching threshold corresponding to the UE;

    The sixth processing unit is configured to receive the beam switching threshold corresponding to the UE sent by the first network node, and perform beam switching based on the beam switching threshold.
31. A processor-readable storage medium, the processor-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 or claim 10 is implemented the method described.