WO2025199714A1

WO2025199714A1 - Devices and methods for communication

Info

Publication number: WO2025199714A1
Application number: PCT/CN2024/083681
Authority: WO
Inventors: Zhen He; Peng Guan; Rao SHI; Gang Wang
Original assignee: NEC Corp
Current assignee: NEC Corp
Priority date: 2024-03-25
Filing date: 2024-03-25
Publication date: 2025-10-02
Anticipated expiration: 2026-09-25

Abstract

Embodiments of the present disclosure provide a solution for providing inaccurate predicted information. In a solution, a first device determines inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event; and transmits at least one message to a second device, the at least one message indicating at least one of the following: the inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information.

Description

DEVICES AND METHODS FOR COMMUNICATION

FIELDS

Example embodiments of the present disclosure generally relate to the field of communication techniques and in particular, to devices and methods for providing inaccurate predicted information.

BACKGROUND

As communication networks and services increase in size, complexity, and number of users, operations in the communication networks may become increasingly more complicated. In order to improve the communication performance, machine learning (ML) /artificial intelligence (AI) technology is proposed to be used in the wireless communication network. For example, the terminal device and the network device may use different ML models to assist communication-related functionalities, such as, beam management (BM) , mobility management and so on.

SUMMARY

In general, embodiments of the present disclosure provide a solution for providing inaccurate predicted information.

In a first aspect, there is provided a first device. The first device comprises: a processor configured to cause the first device to: determine inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event; and transmit at least one message to a second device, the at least one message indicating at least one of the following: the inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information.

In a second aspect, there is provided a second device. The second device comprises: a processor configured to cause the second device to: receive, at least one message from a first device, the at least one message indicating at least one of the following: inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information, wherein the inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event.

In a third aspect, there is provided a communication method performed by a first device. The method comprises: determining inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event; and transmitting at least one message to a second device, the at least one message indicating at least one of the following: the inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information.

In a fourth aspect, there is provided a communication method performed by a second device. The method comprises: receiving, at least one message from a first device, the at least one message indicating at least one of the following: inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information, wherein the inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event.

In a fifth aspect, there is provided a computer readable medium having instructions stored thereon, the instructions, when executed on at least one processor, causing the at least one processor to carry out the method according to the third, or fourth aspect.

Other features of the present disclosure will become easily comprehensible through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

Through the more detailed description of some example embodiments of the present disclosure in the accompanying drawings, the above and other objects, features and advantages of the present disclosure will become more apparent, wherein:

FIG. 1 illustrates an example communication environment in which example embodiments of the present disclosure can be implemented;

FIG. 2A illustrates example beam set for beam prediction in accordance with some embodiments of the present disclosure;

FIG. 2B illustrates another example communication environment in which example embodiments of the present disclosure can be implemented;

FIG. 2C illustrates a signaling flow of communication in accordance with some embodiments of the present disclosure;

FIG. 3 illustrates a flowchart of a communication method implemented at a first device according to some example embodiments of the present disclosure;

FIG. 4 illustrates a flowchart of a communication method implemented at a second device according to some example embodiments of the present disclosure;

FIG. 5 illustrates a simplified block diagram of an apparatus that is suitable for implementing example embodiments of the present disclosure.

Throughout the drawings, the same or similar reference numerals represent the same or similar element.

DETAILED DESCRIPTION

Principle of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. Embodiments described herein can be implemented in various manners other than the ones described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

As used herein, the term ‘terminal device’ refers to any device having wireless or wired communication capabilities. Examples of the terminal device include, but not limited to, user equipment (UE) , personal computers, desktops, mobile phones, cellular phones, smart phones, personal digital assistants (PDAs) , portable computers, tablets, wearable devices, internet of things (IoT) devices, Ultra-reliable and Low Latency Communications (URLLC) devices, Internet of Everything (IoE) devices, machine type communication (MTC) devices, devices on vehicle for V2X communication where X means pedestrian, vehicle, or infrastructure/network, devices for Integrated Access and Backhaul (IAB) , Space borne vehicles or Air borne vehicles in Non-terrestrial networks (NTN) including Satellites and High Altitude Platforms (HAPs) encompassing Unmanned Aircraft Systems (UAS) , eXtended Reality (XR) devices including different types of realities such as Augmented Reality (AR) , Mixed Reality (MR) and Virtual Reality (VR) , the unmanned aerial vehicle (UAV) commonly known as a drone which is an aircraft without any human pilot, devices on high speed train (HST) , or image capture devices such as digital cameras, sensors, gaming devices, music storage and playback appliances, or Internet appliances enabling wireless or wired Internet access and browsing and the like. The ‘terminal device’ can further have ‘multicast/broadcast’ feature, to support public safety and mission critical, V2X applications, transparent IPv4/IPv6 multicast delivery, IPTV, smart TV, radio services, software delivery over wireless, group communications and IoT applications. It may also incorporate one or multiple Subscriber Identity Module (SIM) as known as Multi-SIM. The term “terminal device” can be used interchangeably with a UE, a mobile station, a subscriber station, a mobile terminal, a user terminal or a wireless device.

The term “network device” refers to a device which is capable of providing or hosting a cell or coverage where terminal devices can communicate. Examples of a network device include, but not limited to, a Node B (NodeB or NB) , an evolved NodeB (eNodeB or eNB) , a next generation NodeB (gNB) , a transmission reception point (TRP) , a remote radio unit (RRU) , a radio head (RH) , a remote radio head (RRH) , an IAB node, a low power node such as a femto node, a pico node, a reconfigurable intelligent surface (RIS) , and the like.

The terminal device or the network device may have Artificial intelligence (AI) or Machine learning capability. It generally includes a model which has been trained from numerous collected data for a specific function, and can be used to predict some information.

The terminal or the network device may work on several frequency ranges, e.g., FR1 (e.g., 450 MHz to 6000 MHz) , FR2 (e.g., 24.25GHz to 52.6GHz) , frequency band larger than 100 GHz as well as Tera Hertz (THz) . It can further work on licensed/unlicensed/shared spectrum. The terminal device may have more than one connection with the network devices under Multi-Radio Dual Connectivity (MR-DC) application scenario. The terminal device or the network device can work on full duplex, flexible duplex and cross division duplex modes.

The embodiments of the present disclosure may be performed in test equipment, e.g., signal generator, signal analyzer, spectrum analyzer, network analyzer, test terminal device, test network device, channel emulator. In some embodiments, the terminal device may be connected with a first network device and a second network device. One of the first network device and the second network device may be a master node and the other one may be a secondary node. The first network device and the second network device may use different radio access technologies (RATs) . In some embodiments, the first network device may be a first RAT device and the second network device may be a second RAT device. In some embodiments, the first RAT device is eNB and the second RAT device is gNB. Information related with different RATs may be transmitted to the terminal device from at least one of the first network device or the second network device. In some embodiments, first information may be transmitted to the terminal device from the first network device and second information may be transmitted to the terminal device from the second network device directly or via the first network device. In some embodiments, information related with configuration for the terminal device configured by the second network device may be transmitted from the second network device via the first network device. Information related with reconfiguration for the terminal device configured by the second network device may be transmitted to the terminal device from the second network device directly or via the first network device.

As used herein, the singular forms ‘a’ , ‘an’ and ‘the’ are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term ‘includes’ and its variants are to be read as open terms that mean ‘includes, but is not limited to. ’ The term ‘based on’ is to be read as ‘at least in part based on. ’ The term ‘one embodiment’ and ‘an embodiment’ are to be read as ‘at least one embodiment. ’ The term ‘another embodiment’ is to be read as ‘at least one other embodiment. ’ The terms ‘first, ’ ‘second, ’ and the like may refer to different or same objects. Other definitions, explicit and implicit, may be included below.

In some examples, values, procedures, or apparatus are referred to as ‘best, ’ ‘lowest, ’ ‘highest, ’ ‘minimum, ’ ‘maximum, ’ or the like. It will be appreciated that such descriptions are intended to indicate that a selection among many used functional alternatives can be made, and such selections need not be better, smaller, higher, or otherwise preferable to other selections.

As used herein, the term “resource, ” “transmission resource, ” “uplink resource, ” or “downlink resource” may refer to any resource for performing a communication, such as a resource in time domain, a resource in frequency domain, a resource in space domain, a resource in code domain, or any other resource enabling a communication, and the like. In the following, unless explicitly stated, a resource in both frequency domain and time domain will be used as an example of a transmission resource for describing some example embodiments of the present disclosure. It is noted that example embodiments of the present disclosure are equally applicable to other resources in other domains.

As used herein, the terms “UE expects” , “UE does not expect, “terminal device expects” , “terminal device does not expect” may imply restrictions on a configuration of a network device (also referred to as NW configuration) . The terms “UE is not expected to” and “terminal device is not expected to” may imply a terminal implementation, also referred to as UE implementation. In some embodiments, the terms “UE does not expect” and “UE is not expected to” may be used equally.

As discussed above, the terminal device and the network device may use different ML models to assist communication-related functionalities, such as, beam management (BM) , mobility management and so on.

So far, two BM cases (i.e., BM-case1 and BM-case2) have been proposed and discussed separately, specifically,

● BM-Case1: Spatial-domain downlink beam prediction for Set A of beams based on measurement results of Set B of beams;

Consider: 1) : artificial intelligence (AI) /machine learning (ML) model training and inference at NW side. 2) : AI/ML model training and inference at UE side.

Consider: 1) : Set A and Set B are different (Set B is not a subset of Set A) . 2) : Set B is a subset of Set A. Note: Set A is for DL beam prediction.

AI/ML model input consider: 1) : only L1-RSRP measurement based on Set B; 2) : L1-RSRP measurement based on Set B and assistance information; 3) : channel impulse response (CIR) based on Set B; 4) : layer 1 (L1) -reference signal receiving power (RSRP) measurement based on Set B and the corresponding DL Tx and/or Rx beam ID.

● BM-Case2: Temporal Downlink beam prediction for Set A of beams based on the historic measurement results of Set B of beams;

Consider: 1) : AI/ML model training and inference at NW side. 2) : AI/ML model training and inference at UE side.

Consider: 1) : Set A and Set B are different (Set B is NOT a subset of Set A) . 2) : Set B is a subset of Set A (Set A and Set B are not the same) . 3) : Set A and Set B are the same.

AI/ML model input consider: measurement results of K (K≥1) latest measurement instances with the following alternatives: 1) : Only L1-RSRP measurement based on Set B; 2) : L1-RSRP measurement based on Set B and assistance information; 3) : L1-RSRP measurement based on Set B and the corresponding DL Tx and/or Rx beam ID.

F predictions for F future time instances can be obtained based on the output of AI/ML model, where each prediction is for each time instance. At least F=1.

Set B is a set of beams whose measurements may be taken as inputs of the AI/ML model.

It has been agreed to provide specification support for the following aspects:

● Beam management -DL Tx beam prediction for both UE-sided model and NW-sided model, encompassing:

- Spatial-domain DL Tx beam prediction for Set A of beams based on measurement results of Set B of beams ( “BM-Case1” ) ;

- Temporal DL Tx beam prediction for Set A of beams based on the historic measurement results of Set B of beams ( “BM-Case2” ) ;

- Specify necessary signalling/mechanism (s) to facilitate LCM operations specific to the Beam Management use cases, if any;

- Enabling method (s) to ensure consistency between training and inference

regarding NW-side additional conditions (if identified) for inference at UE.

● Strive for common framework design to support both BM-Case1 and BM-Case2.

Regarding the AI/ML for mobility in NR, the study will focus on mobility enhancement in radio resource control (RRC) _CONNECTED mode over air interface by following existing mobility framework, i.e., handover decision is always made in network side. Mobility use cases focus on standalone NR PCell change. UE-side and network-side AI/ML model can be both considered, respectively.

Further, it also needs to study and evaluate potential benefits and gains of AI/ML aided mobility for network triggered layer 3 (L3) -based handover, considering the following aspects:

● AI/ML based RRM measurement and event prediction,

- Cell-level measurement prediction including intra and inter-frequency (UE-sided and NW-sided model) . Inter-cell Beam-level measurement prediction for L3 Mobility (UE sided and NW sided model) ;

- Handover (HO) failure/radio link failure (RLF) prediction (UE sided model) ;

- Measurement events prediction (UE sided model) ;

● Study the need/benefits of any other UE assistance information for the network side model;

● Potential AI mobility specific enhancement should be based on the AI/ML-air interface work item description (WID) general framework (e.g. life cycle management (LCM) , performance monitoring and so on) .

For better descriptions, some terms used herein are listed as below:

Beam can be replaced or represented by beam ID, where the beam ID is interchangeably with (Tx/Rx) beam ID, RS (e.g., Channel state information reference signal, CSI-RS, synchronization signal or physical broadcast channel (PBCH) Block, SSB, sounding reference signal, SRS) ID, RS resource ID (e.g., CSI-RS resource indicator (CRI) , SRS resource indicator (SSBRI) ) , transmission configuration indicator (TCI) state ID, quasi co-location (QCL) RS (e.g., QCL-typeD RS) ID, measurement RS ID, measurement RS resource ID, path-loss RS ID. Beam may be interchangeably with (Tx/Rx) beam, beam direction/orientation, RS (e.g., CSI-RS, SSB, SRS) , RS resource, TCI state, QCL RS (e.g., QCL-typeD RS) , measurement RS, measurement RS resource, path-loss RS.

Measurement result (s) /beam quality may include but be not limited to, L1-reference signal received power (RSRP) , L1-SINR, L1-received signal strengthen indicator (RSSI) , L1-reference signal received quality (RSRQ) , receive signal channel power (RSCP) , RSRP, SINR, RSSI, or RSRQ.

Predicted L1-RSRP means a predicted quality/metric related to beam, cell or measurement event, and/or outputted by the AI/ML model related to beam prediction, cell prediction or measurement event prediction.

RRC message is a L3 signaling. MAC CE is a L2 signaling. Uplink control information (UCI) is a L1 signaling. Each of them may be transmitted in PUSCH or/and physical uplink control channel (PUCCH) .

Measurement report initiated by UE means that a measurement report triggered or initiated by UE based on certain pre-defined or NW configured event (s) .

Indicator of set may refer to RS resource set ID, where each RS resource in the RS resource corresponds to a beam.

Model inference means a process of using a trained AI/ML model to produce a set of outputs based on a set of inputs.

Model training means a process to train an AI/ML Model [by learning the input/output relationship] in a data driven manner and obtain the trained AI/ML Model for inference.

Model switching means deactivating a currently active AI/ML model and activating a different AI/ML model for a specific AI/ML-enabled feature.

Model selection means a process of selecting an AI/ML model for activation among multiple models for the same AI/ML enabled feature.

Model update means a process of updating the model parameters and/or model structure of a model.

Model activation means to enable an AI/ML model for a specific AI/ML-enabled feature.

Model monitoring means a procedure that monitors the inference performance of the AI/ML model.

Reference data distribution means a data distribution corresponding to a dataset used for training/validating/testing/updating an AI/ML model.

Similarity or dissimilarity (or diversity) may be a value that is used to represent/reflect the difference between a data distribution and another data distribution. It may comprise at least one of correlation coefficient, cosine similarity, Kullback-Leibler divergence, Euclidean distance, Manhattan distance, Jaccard similarity coefficient, etc.

Value representing a data distribution may be (but not be limited to) PDF (probability density function) or CDF (cumulative distribution function) . For example, if the data distribution is CDF, the value can be the probability of being less than or larger than a certain RSRP related threshold, or the RSRP value of being less than or greater than a certain probability.

Top-1 beam means the beam having the largest beam (measured or predicted) quality (e.g., L1-RSRP, L1-SINR) in a set of beams.

Top-1 cell means the cell having the largest (measured or predicted) quality (e.g., RSRP, SINR, RSRQ) in a set of cells, it can be interchangeably with target cell.

The actual top-1 beam/cell is interchangeably with real/effective/practical/ideal top-1 beam, top-1 genie-aided beam.

Data sample is interchangeably with data sample, sample.

Period of time means a time duration where performance monitoring is performed. And it is interchangeably with time window, time duration, timer, monitoring window.

In the context of the present disclosure,

terms “beam” may be replaced by “beam pair” ;

terms “ID” , “identifier” , “identity” , “index” or “indicator” , “indication” may be used interchangeably;

Occur may be replaced with declare, happen, take place.

Condition may be replaced with (pre-defined) condition, event, rule, criterion.

Accurate may be replaced with precise, good, success, correct, right, exact, true.

Inaccurate may be replaced with incorrect, inexact, improper, wrong, not good, bad, failure, false.

Configure may be replaced with indicate, provide, activate, trigger.

Probability may be replaced with proportion, ratio, confidence (level/interval) .

Report may be replaced with transmit, send, provide, indicate.

Terminal device/UE (or network device/NW) may be replaced with OTT (server) , OAM (server) , CN (server) , edge cloud (server) , transmission reception point (TRP) ; Predicted time instance may be replaced with (future or predicted) time instance, time point, time stamp, time interval, time.

Cell may be replaced with serving/non-serving/source/target/candidate cell, (active or inactive) DL/UL BWP, frequency range, frequency carrier, carrier component (CC) , cell group, PCell, SCell, PScell, (cell) configuration for mobility/measurement/report (e.g., LTM/CLTM cell configuration) . Cell may be replaced or represented by indicator of cell, where the indicator of cell may refer to PCI (physical cell identifier) , serving cell index, SSB index, cell ID, LTM candidate ID, measurement ID, report ID.

Zone may be replaced with region, site, area, sector, location. And it may comprise one or multiple cells (or/and partial regions of cell) .

Predict may be replaced with inference, output, estimate.

As used herein, term “specific” means predefined, predetermined, particular, or unique. For example, a specific beam may refer to a predefined beam, a predetermined beam, a particular beam, or a unique beam.

As used herein, model may refer to AI/ML, AI/ML model, functionality, AI/ML functionality, AI-enabled feature/feature group (FG) , which means a data driven algorithm that applies AI/ML techniques to generate a set of (AI/ML) outputs based on a set of (AI/ML) inputs. AI/ML-enabled feature refers to a feature where AI/ML may be used.

Model ID may be one of the following: functionality ID, dataset ID, scenario ID, zone ID, configuration ID, quantization ID, local (model) ID, global (model) ID, logical (model) ID, physical (model) ID, etc.

In some embodiments, an input of the ML model (i.e., AI input) may refer to the input of a model and indicate data inputted into the model, which may be equivalent to data.

In some embodiments, an output of ML model (i.e., AI output) may refers to the output of a model and indicate result (s) outputted by the model, which is equivalent to label/data.

In some embodiments, the model may comprise a set of weights values that may be learned during training, for example for a specific architecture or configuration, where a set of weights values may also be called a parameter set.

Wording ‘Acorresponds to B’ may be replaced by ‘Ais associated with B’ , ‘Ais mapped to B’ , or ‘B is mapped to A’ . Term ‘confidence’ may be replaced by uncertainty, reliability, probability, confidence level.

It should be noted that, examples where the inaccurate predicted information are provided. As the accurate predicted information and inaccurate predicted information are corresponding with each other, and thus all the examples related to inaccurate predicted information also may be applicable to the cases of providing accurate predicted information. Merely for brevity, the same or the similar contents are omitted herein.

Principles and implementations of the present disclosure will be described in detail below with reference to the figures.

Example environment

FIG. 1 illustrates a schematic diagram of an example communication environment 100 in which example embodiments of the present disclosure can be implemented. In the communication environment 100, a plurality of communication devices, including a first device 110 and a second device 120, can communicate with each other.

Further, multiple input multiple output (MIMO) is supported in the communication environment 100, such that the second device 120 and the first device 110 may communicate with each other via different beams to enable a directional communication.

In FIG. 1, the first device 110/second device 120 may be included a terminal device, a network device, an over the top (OTT) (server) , an operation administration and maintenance (OAM) (server) , an edge cloud (server) , a neutral site, core network, transmission reception point (TRP) and so on.

As one example scenario, the first device 110 may include a terminal device and the second device 120 may include a network device serving the terminal device. In this specific example embodiment, a link from the first device 110 to the second device 120 is referred to as uplink, while a link from the second device 120 to the first device 110 is referred to as a downlink.

In downlink, the second device 120 is a transmitting (TX) device (or a transmitter) and the first device 110 is a receiving (RX) device (or a receiver) , and the second device 120 may transmit downlink transmission to the first device 110 via one or more beams. As illustrated in FIG. 1, the second device 120 transmits downlink transmission to the first device 110 via the beams 140-1 to 140-3. For purpose of discussion, the beams 140-1 to 140-3 are collectively or individually referred to as beam 140.

Correspondingly, in uplink, the second device 120 is an RX device (or a receiver) and the first device 110 is a TX device (or a transmitter) , and the first device 110 may transmit uplink transmission to the second device 120 via one or more beams. As illustrated in FIG. 1, the first device 110 transmits uplink transmission to the second device 120 via the beams 130-1 to 130-3. For purpose of discussion, the beams 130-1 to 130-3 are collectively or individually referred to as beam 130.

In some embodiments, one or more models may be deployed at the second device 120 and/or the first device 110. As illustrated in FIG. 1, the model 115 is deployed at the first device 110.

In some embodiments, performance monitoring of the models needs to be performed. The following metrics/methods for AI/ML model monitoring in lifecycle management per use case are considered:

● monitoring based on inference accuracy, including metrics related to intermediate Key Performance Indicators (KPIs) ;

● monitoring based on system performance, including metrics related to system performance KPIs;

● monitoring based on data distribution, input-based: e.g., monitoring the validity of the AI/ML input, e.g., out-of-distribution detection, drift detection of input data, or SNR, delay spread and so on;

● monitoring based on data distribution, output-based: e.g., drift detection of output data;

● monitoring based on applicable condition.

It should be noted that the monitoring metric calculation may be done at the NW or UE.

Methods to assess/monitor the applicability and expected performance of an inactive model/functionality, including the following examples for the purpose of activation/selection/switching of UE-side models/UE-part of two-sided models /functionalities (if applicable) :

● assessment/monitoring based on the additional conditions associated with the model/functionality;

● assessment/monitoring based on input/output data distribution;

● assessment/monitoring using the inactive model/functionality for monitoring purpose and measuring the inference accuracy;

● assessment/monitoring based on past knowledge of the performance of the same model/functionality (e.g., based on other UEs) .

For the performance monitoring of BM-Case1 and BM-Case2, the performance metric (s) with the following alternatives: beam prediction accuracy related KPIs, e.g., Top-K/1 beam prediction accuracy; link quality related KPIs, e.g., throughput, (L1) -RSRP, (L1) -SINR, hypothetical block error rate (BLER) ; performance metric based on input/output data distribution of AI/ML; and the (L1) -RSRP difference evaluated by comparing measured RSRP and predicted RSRP.

For AI/ML models, which provide L1-RSRP as the model output, to evaluate the accuracy of predicted L1-RSRP, companies optionally report average (absolute value) /cumulative distribution function (CDF) of the predicted L1-RSRP difference, where the predicted L1-RSRP difference is defined as: the difference between the predicted L1-RSRP of Top-1 [/K] predicted beam and the ideal L1-RSRP of the same beam.

In the FIG. 1, regression-based model and classification-based model may be supported. In a case of regression-based model, below prediction may be enabled:

● BM-Case1/2 (spatial/temporal beam prediction) , i.e., beam-level measurement prediction, where, output may be L1-RSRP of beam, in a set of beams, and input may be L1-RSRP of beam, in a further set of beams.

● (Spatial/temporal) cell-level measurement prediction, where, output may be RSRP, SINR, or RSRQ of cell, in a set of cells, input may be RSRP, SINR, or RSRQ of cell, in a further set of cells.

In a case of classification-based model, below prediction may be enabled:

● BM-Case1/2 (spatial/temporal beam prediction) , i.e., beam-level measurement prediction, where, output may be (indicator of) top-1 beam, in a set of beams, and input may be L1-RSRP of beam, in a further set of beams;

● (Target) cell prediction, output may be (indicator of) target cell and input may be RSRP, SINR, or RSRQ of cell, in a further set of cells, and/or L1-RSRP or L1-SINR of beam, in a set of beams associated with the cell in the further set;

● Measurement event prediction, where, output may be (indicator of) measurement event, and input may be: RSRP, SINR, or RSRQ of cell, in a further set of cells, and/or L1-RSRP or L1-SINR of beam, in a set of beams associated with the cell in the further set.

It is to be understood that the number of devices and their connections shown in FIG. 1 are only for the purpose of illustration without suggesting any limitation. The communication environment 100 may include any suitable number of devices configured to implementing example embodiments of the present disclosure.

In some embodiments, the first device 110 and the second device 120 may communicate with each other via a channel such as a wireless communication channel on an air interface (e.g., Uu interface) . The wireless communication channel may comprise a physical uplink control channel (PUCCH) , a physical uplink shared channel (PUSCH) , a physical random-access channel (PRACH) , a physical downlink control channel (PDCCH) , a physical downlink shared channel (PDSCH) and a physical broadcast channel (PBCH) . Of course, any other suitable channels are also feasible.

The communications in the communication environment 100 may conform to any suitable standards including, but not limited to, Global System for Mobile Communications (GSM) , Long Term Evolution (LTE) , LTE-Evolution, LTE-Advanced (LTE-A) , New Radio (NR) , Wideband Code Division Multiple Access (WCDMA) , Code Division Multiple Access (CDMA) , GSM EDGE Radio Access Network (GERAN) , Machine Type Communication (MTC) and the like. The embodiments of the present disclosure may be performed according to any generation communication protocols either currently known or to be developed in the future. Examples of the communication protocols include, but not limited to, the first generation (1G) , the second generation (2G) , 2.5G, 2.75G, the third generation (3G) , the fourth generation (4G) , 4.5G, the fifth generation (5G) communication protocols, 5.5G, 5G-Advanced networks, or the sixth generation (6G) networks.

Example processes

As discussed above, performance monitoring of the model is needed. Regarding performance monitoring for (UE-side) AI/ML model related to beam prediction, cell prediction and measurement event prediction, in some cases, the degradation in prediction accuracy may be confined solely to partial beams/cells/measurement events rather all beams/cells/measurement events.

Reference is now made to FIG. 2A and FIG. 2B, where FIG. 2A illustrates example beam set 200A for beam prediction in accordance with some embodiments of the present disclosure, and FIG. 2B illustrates another example communication environment 200B in which example embodiments of the present disclosure can be implemented.

In the example of FIG. 2A, an input beam set (also called as Set A) consists of 6 beams: beam-4, beam-9, beam-14, beam-19, beam-24, beam-29, and an output beam set (also called as Set B) consists of 32 beams: beam-0, beam-1, ..., beam-31. Further refer to FIG. 2B, some beams (e.g., beam-12) may be affected or blocked due to unexpected or unpredictable obstacle. In this case, in the set of predicted beams (i.e., Set A) , only beam-12 may not be accurately predicted by the AI/ML model. Specifically, for regression-based AI/ML model, the predicted L1-RSRP of beam-12 is inaccurate. For classification-based AI/ML model, the AI/ML model cannot accurately identify beam-12 as the top-1 beam. In other words, only when the AI/ML model outputs beam-12 as the top-1 beam can it be considered that this predicted result is inaccurate.

In view of this, during performance monitoring, it is possible that only a portion of the beams (or cells, measurement events) cannot be accurately predicted, rather than all beams. If UE fails to report these beam information (i.e., inaccurate predicted beams) to NW, it may result in inaccurate prediction results being used or unnecessary model update, model switching, or fallback (to non-AI/ML based beam management) . Therefore, UE needs to provide inaccurate predicted beams to the NW. However, there is currently no method established on how to determine and report such information.

Reference is made to FIG. 2C, which illustrates a signaling flow 200C for communication in accordance with some embodiments of the present disclosure. For the purposes of discussion, the signaling flow 200C will be discussed with reference to FIG. 1, for example, by using the first device 110 and the second device 120.

It is to be understood that the operations at the first device 110 and the second device 120 should be coordinated. In other words, the second device 120 and the first device 110 should have common understanding about configurations, parameters and so on. Such common understanding may be implemented by any suitable interactions between the second device 120 and the first device 110 or both the second device 120 and the first device 110 applying the same rule/policy. In the following, although some operations are described from a perspective of the first device 110, it is to be understood that the corresponding operations should be performed by the second device 120. Similarly, although some operations are described from a perspective of the second device 120, it is to be understood that the corresponding operations should be performed by the first device 110. Merely for brevity, some of the same or similar contents are omitted here.

For the purpose of discussion, the first device 110 may be a terminal device and the second device 120 may be a network device. It should be understood that, in the other embodiments, the first device 110/second device 120 may be any of: a terminal device, a network device, an over the top (OTT) (server) , an operation administration and maintenance (OAM) (server) , an edge cloud (server) , a neutral site, transmission reception point (TRP) core network and so on. In present disclosure is not limited in this regard.

In the present disclosure, an element may be a beam, a cell, an event or any other element which may be predicted by an ML mode. Further, the inaccurate prediction information may be related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event. In view of this, terms of beam, cell and event may be used interchangeably.

As illustrated in FIG. 2C, in operation, the first device 110 determines (250) inaccurate prediction information associated with at least one machine learning (ML) model at the first device 110, where the inaccurate prediction information is related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event.

Then the first device 110 transmits (260) at least one message to a second device 120, the at least one message indicating at least one of the following:

● the inaccurate prediction information,

● a partial model failure determined based on the inaccurate prediction information, or

● a full model failure determined based on the inaccurate prediction information.

In some embodiments, the at least one message may comprise at least one of the following: radio resource control (RRC) signalling, a medium access control (MAC) control element (CE) or uplink control information (UCI) .

In some embodiments, the at least one message may comprise at least one of a measurement report, a user equipment (UE) assistance information (UAI) , or a UE capability information.

In some embodiments, the at least one message may be transmitted on at least one of a PUCCH resource or a PUSCH resource associated with a predefined scheduling request (SR) .

As discussed above, both regression-based model and classification-based model may be supported and the outputs of these two types of model are different. As a result, different determination procedures and reporting procedures are needed for these two types of model.

Examples about the regression-based model will be discussed first. In some embodiments, the first device 110 may determine a first set of measurement results of a first set of elements, where the first set of elements is a first set of beams or a first set of cells. Further, the first device 110 may determine, by the at least one ML model, a first set of prediction results of the first set of elements.

In some embodiments, the first set of elements may be a set of beams associated with at least one output of the at least one ML model. Alternatively, or in addition, in some embodiments, the first set of elements is a set of cells associated with at least one output of the at least one ML model. Alternatively, or in addition, in some embodiments, the first set of elements is configured by the second device 120. Alternatively, or in addition, in some embodiments, the first set of elements is determined based on a pre-predefined criterion. Alternatively, or in addition, in some embodiments, the first set of elements is determined based on a pre-predefined threshold.

After that, for each element in the first set, the first device 110 may determine whether the element is an inaccurate predicted element based on at least one first difference, where each first difference is determined based on a measurement result of the element and a prediction result of the element.

In this event, in some embodiments, the element may be determined to be an inaccurate predicted element if at least one of the following:

● a first difference between a measurement result of the element and a prediction result of the element is equal to or larger than a first threshold,

● a number of times that the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold is equal to or larger than a second threshold,

● the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold over a first period of time.

Alternatively, or in addition, in some embodiments, the first device 110 may determine whether the element is an inaccurate predicted element based on at least one statistical value of first differences. Additionally, in some embodiments, the statistical value may be one of the following: a mean value, a maximum value, a minimum value, a median value, a variance, or a standard deviation.

● a statistical value of first differences of the element is equal to or larger than a third threshold,

● a number of times that the statistical value of first differences of the element is equal to or larger than a third threshold is equal to or larger than a fourth threshold, or

● the statistical value of first differences of the element is equal to or larger than a third threshold over a second period of time.

Additionally, additional criterion may be applied when determining the inaccurate predicted element. Specifically, in some embodiments, the element may be determined to be an inaccurate predicted element if at least one of the following:

● a measurement result of the element is equal to or smaller than a fifth threshold,

● a number of times that the measurement result of the beam or cell is equal to or smaller than a fifth threshold is equal to or larger than a sixth threshold, or

● the measurement result of the beam or cell is equal to or smaller than a fifth threshold over a third period of time.

Alternatively, or in addition, in some embodiments, the element may be determined to be an inaccurate predicted element if at least one of the following:

● a prediction result of the beam or cell is equal to or larger than a seventh threshold,

● a number of times that the prediction result of the beam or cell is equal to or larger than a seventh threshold is equal to or larger than an eighth threshold, or

● the prediction result of the beam or cell is equal to or larger than a seventh threshold over a fourth period of time.

Examples about the classification-based model will be discussed in the following.

In some embodiments, for an element in a set of elements, the first device 110 may determine whether the element is an inaccurate predicted element based on the following: at least one comparation result associated with the element, each comparation result indicating whether a first order of the element in a set of measurement results is consistent with a second order of the element in a set of prediction results. In this event, in some embodiments, the element may be determined to be an inaccurate element if at least one of the following:

● the comparation result indicates that the first order of the element is inconsistent with the second order of the element,

● a number of times that the comparation result indicates that the first order of the element is inconsistent with the second order of the element is equal to or larger than a first threshold, or

● the comparation result indicates that the first quality order of the element is inconsistent with the second quality order over a first period of time.

As for the classification -based model, in some embodiments, for an element in a set of elements, the first device 110 may determine whether the element is an inaccurate predicted element based on at least one first data distribution associated with the element (first data distribution corresponding to a set of measurement results associated with the element) .

● a value representing the first data distribution of the element is larger than or equal to a threshold a first threshold,

● a number of times that the value representing the first data distribution of the element is larger than or equal to a threshold a first threshold is equal to or larger than a second threshold,

● the value representing the first data distribution of the element is larger than or equal to a first threshold over a first period.

Alternatively, or in addition, in some embodiments, for an element in a set of elements, the first device 110 may determine whether the element is an inaccurate predicted element based on at least one second difference associated with the element, wherein each second difference is determined based on a first data distribution associated with the element and a reference data distribution.

● the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third threshold,

● a number of times that the second difference is equal to or larger than a third threshold is equal to or larger than a fourth threshold,

● the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third fourth over a third period of time.

According to example embodiments of the present disclosure, based on the inaccurate prediction information, the first device 110 may further determine whether a partial model failure occurs or a full model failure occurs as discussed below.

In some embodiments, the first device 110 may determine a partial model failure occurs if at least one of the following:

● a number of detected inaccurate predicted elements is equal to or smaller than a threshold, or

● a number of times of detecting an inaccurate predicted element is equal to or smaller than a threshold.

Alternatively, or in addition, in some embodiments, the first device 110 may determine a full model failure occurs if at least one of the following:

● a number of detected inaccurate predicted elements is equal to or larger than a threshold, or

● a number of times of detecting an inaccurate predicted element is equal to or larger than a threshold.

In the following, details about the contents comprised in the at least one message will be discussed.

In some embodiments, the inaccurate prediction information (comprised in the at least one message) may comprise at least one of the following:

● at least one indicator of the at least one inaccurate predicted beam,

● at least one indicator of the at least one inaccurate predicted cell, or

● at least one indicator of the at least one inaccurate predicted event.

Alternatibely, the first device 110 may provide more information to the second device 120, such that the second device 120 may understand more details about the prediction performance.

In some embodiments, the at least one message indicates at least one of the following:

● a number of detected inaccurate predicted elements,

● a number of times of detecting an inaccurate predicted element,

● a statistical value of first differences associated with an inaccurate predicted element,

● a first difference between a measurement result of an inaccurate predicted element and a prediction result of the inaccurate predicted element,

● a second difference between a first data distribution associated with an inaccurate predicted element and a reference data distribution,

● a value representing the second difference,

● a type of the first data distribution associated with an inaccurate predicted element,

● a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is inconsistent with a second quality order of the element determined based on a set of prediction results,

● a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is consistent with a second quality order of the element determined based on a set of prediction results,

● an indication used for requesting a re-configuration of a set of beams or cells associated with an output of the at least one ML model,

● an indication used for requesting a re-configuration of a set of cells associated with an output of the at least one ML model,

● an identity of a set of beams associated with the inaccurate predicted element,

● an identity of a set of cells associated with the inaccurate predicted element,

● an indicator of a predicted time instance associated with the inaccurate predicted element,

● an indicator of a cell associated with the inaccurate predicted element,

● an indicator of a zone associated with the inaccurate predicted element, or

● an identity of the at least one ML model associated with the inaccurate predicted element.

In this way, based on the inaccurate (or accurate) predicted information determined and reported by the first device 110, the second device 120 may configure reasonable measurement resources and prediction resources (or available predicted element) . For example, the second device 120may configure measurement resource for the inaccurate predicted element reported by the first device 110, so that the first device 110 can obtain the actual measured L1-RSRP of the inaccurate predicted element.

In addition, unnecessary entire model update/switching/fallback (to non-AI/ML mode) may be avoided. Further, it is beneficial for reasonable model update, e.g., unnecessary overhead (i.e., data required for model update) and latency of model update may be reduced.

Embodiments

In order to better understanding the above processes, some example embodiments will be further discussed, where a UE is used as an example of the first device and a NW is used as an example of the second device. According to the present disclosure, inaccurate predicted beam/cell/measurement event may be determined and reported, and/or a concept of partial model failure is introduced.

Example embodiments of regression based model and beam-level measurement prediction will be discussed in the following, where the monitoring may be performed based on L1-RSRP/RSRP difference.

In case of spatial beam prediction (e.g., using regression-based model) , the NW may configure for UE a set of beams (e.g., Set A) and a further set of beams (e.g., Set B) , which may be associated with at least one AI/ML model. Specifically, the set of beams may be used as predicted beams in model inference, i.e., the output (s) of the at least one AI/ML model may derive from the set of beams. The further set of beams may be used as measured beams in model inference, i.e., the input (s) of the at least one AI/ML model may derive from the measurements of the further set of beams.

UE may calculate qualities (e.g., L1-RSRPs) of all beams in the set (i.e., determine measured L1-RSRPs of all beams in the set) , and determine measured L1-RSRPs of all beams in the further set. Then, UE may determine predicted L1-RSRPs of all beams in the set based on the at least one AI/ML model and the measured L1-RSRPs of all beams in the further set.

Optionally, UE may determine measured L1-RSRPs and predicted L1-RSRPs of partial beams (i.e., not all beams) in the set, and the partial beams may be determined based on NW’s configuration/indication, or determined based on certain pre-predefined criterion/threshold (e.g., top K (K≥1) beams in the set) .

For each beam in the set, UE may determine a L1-RSRP difference between the measured L1-RSRP and the predicted L1-RSRP corresponding to the beam.

Furthermore, for each beam, UE may determine multiple (consecutive) L1-RSRP differences corresponding to the beam over a period of time. Optionally, for each beam, UE may determine at least one statistical L1-RSRP difference. The at least one statistical L1-RSRP difference may comprise at least one of a mean value, maximum, minimum, median value, variance or standard deviation etc. of the multiple L1-RSRP differences or a value representing data distribution of the multiple L1-RSRP differences.

For each beam in the set (or partial beams in the set, such as, Top K beams) , UE may determine whether the beam is an inaccurate (or failure) predicted beam (optionally, determine whether the beam is an accurate predicted beam) based on the at least one L1-RSRP difference or/and the at least one statistical L1-RSRP difference corresponding to the beam. Specifically, for a given beam, if at least one of the following conditions is fulfilled, (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

(Condition 1) In some embodiments, if the L1-RSRP difference corresponding to the beam is larger than or equal to a threshold, or/and L1-RSRP difference corresponding to the beam is smaller than or equal to a threshold. Alternatively, in some embodiments, the L1-RSRP difference corresponding to the beam is larger than or equal to a threshold over a period of time (i.e., the at least one L1-RSRP difference corresponding to the beam determined during the period of time are larger than or equal to a threshold) , or/and L1-RSRP difference corresponding to the beam is smaller than or equal to a threshold over a period of time, the beam is an inaccurate predicted beam (or accurate predicted beam) .

(Condition 2) In some embodiments, if the measured L1-RSRP of the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the beam is an inaccurate predicted beam (or accurate predicted beam) .

(Condition 3) In some embodiments, if the predicted L1-RSRP of the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the beam is an inaccurate predicted beam (or accurate predicted beam) .

Additionally, in some embodiments, if the number of times (or probability of) at least one of Condition 1, 2 or 3 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the beam is an inaccurate predicted beam (or accurate predicted beam) .

Additionally, in some embodiments, if the at least one statistical L1-RSRP difference corresponding to the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the beam is an inaccurate predicted beam (or accurate predicted beam) .

Furthermore, (if there is at least one inaccurate predicted beam (or accurate predicted beam) ) , UE may determine whether a partial (or incomplete) model failure occurs, or/and a (full, complete or entire) model failure occurs based on the at least one inaccurate predicted beam (or accurate predicted beam) .

In some embodiments, if the number (or probability) of inaccurate predicted beams (or accurate predicted beams) is larger than or equal to a threshold (e.g., 1, or indicated by NW) (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume) that a partial model failure occurs.

Alternatively, or in addition, in some embodiments, if the number of times (or probability of) the above condition is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume) that a partial model failure occurs.

In some embodiments, if the number (or probability) of inaccurate predicted beams (or accurate predicted beams) is larger than or equal to a threshold (e.g., equal to the total number of beams in the set, or indicated by NW) (over a period of time) , (UE may assume) that a (full, complete or entire) model failure occurs.

Alternatively, or in addition, in some embodiments, if the number of times (or probability of) the above condition is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume) that a (full, complete or entire) model failure occurs.

In some embodiments, (if there is at least one inaccurate predicted beam (or accurate predicted beam) ) , UE may report at least one of the following pieces of information (called as ‘first information’ ) to NW in at least one UL message/signaling.

● Indictor of inaccurate predicted beam (or accurate predicted beam) .

● L1-RSRP difference corresponding to inaccurate predicted beam (or accurate predicted beam) .

● Statistical L1-RSRP difference corresponding to inaccurate predicted beam (or accurate predicted beam) .

● The number of times at least one of Condition 1, 2 or 3 is fulfilled.

● Probability of at least one of Condition 1, 2 or 3 is fulfilled.

● The number of inaccurate predicted beams (or accurate predicted beams) .

● Probability of inaccurate predicted beams (or accurate predicted beams) .

● Information indicating whether a partial model failure occurs.

● Information indicating whether a model failure occurs.

In some embodiments, the at least one UL message/signaling may comprise at least one of the following: UL RRC message, a UL MAC CE, or a UL control information (UCI) . Specifically, at least one of the following methods may be adopted for reporting at least one of the first information.

In some embodiments, UE may report at least one of the first information to NW in at least one measurement report (e.g., periodic, semi-persistent or aperiodic report configured by NW, or report initiated by UE) , which is carried by at least one of the above UL message/signaling. For example, the measurement report may be a CSI report.

Alternatively, in some embodiments, UE may report at least one of the first information to NW in an UL RRC message including UE capability information or UE assistance information (UAI) .

Alternatively, in some embodiments, UE may be provided by NW with a dedicated PUCCH resource and/or dedicated scheduling request (SR) (ID) for partial model failure or model failure, where the dedicated SR is used to request UL PUSCH resource for reporting at least one of the first information.

Alternatively, in some embodiments, UE may report at least one of the first information to NW in a request for at least one of: data collection, model update (e.g., fine-tuning, retraining) or/and model training, removing/updating/changing (available) predicted beam or predicted beam in the set of beams (i.e., Set A) ) .

Optionally, in some embodiments, the request may be for removing/updating/changing (available) predicted cell or measurement event or predicted cell or measurement event in the set of cells or measurement events.

In addition to the first information (e.g., the indictor of inaccurate predicted beam (or accurate predicted beam) ) , UE may report at least one of the following pieces of information (maybe together with the first information) :

● Indicator of set, e.g., the set comprising (or corresponding to) the inaccurate predicted beam (or accurate predicted beam) ;

● Indicator of AI/ML model, e.g., the AI/ML model for determining predicted L1-RSRP corresponding to the inaccurate predicted beam (or accurate predicted beam) .

● Indicator of predicted time instance, e.g., the predicted time instance corresponding to the inaccurate predicted beam (or accurate predicted beam) . Specifically, in case of temporal beam prediction, UE may be configured with a set of predicted time instance, and each predicted time instance may be associated with a set of beams (these sets may be the same) ; and

● Indicator of cell, e.g., the cell associated with the inaccurate predicted beam (or accurate predicted beam) . Specifically, in case of inter-cell (spatial/temporal) beam prediction, UE may be configured with a set of cells, and each cell may be associated with a set of beams.

Specifically, the above information may be used to indicate at least one of the following (but not limited to) : there is at least one inaccurate predicted beam (or accurate predicted beam) for the set, AI/ML model, predicted time instance or cell indicated by the above information, and/or whether a partial model failure or model failure occurs for the set, AI/ML model, predicted time instance or cell indicated by the above information.

Based on the inaccurate (or accurate) predicted beam determined and reported by UE, NW can configure reasonable beam measurement resources and beam prediction resources (or available predicted beams) . For example, NW may configure measurement resource for the inaccurate predicted beam reported by UE, so that UE can obtain the actual measured L1-RSRP of the inaccurate predicted beam.

In addition, unnecessary entire model update/switching/fallback (to non-AI/ML based beam prediction) can be avoided. Further, it is beneficial for reasonable model update, e.g., unnecessary overhead (i.e., data required for model update) and latency of model update can be reduced.

Example embodiments of regression based model and cell-level measurement prediction will be discussed in the following, where the monitoring may be performed based on L1-RSRP/RSRP difference.

In operation, the NW may configure for UE a set of cells and a further set of cells, which may be associated with at least one AI/ML model. Specifically, the set of cells may be used as predicted cells in model inference, i.e., the output (s) of the at least one AI/ML model may derive from the set of cells. The further set of cells may be used as measured cells in model inference, i.e., the input (s) of the at least one AI/ML model may derive from the measurements of the further set of cells.

Then, UE may calculate qualities (e.g., RSRPs) of all cells in the set (i.e., determine measured RSRPs of all cells in the set) , and determine measured RSRPs of all cells in the further set. Then, UE may determine predicted RSRPs of all cells in the set based on the at least one AI/ML model and the measured RSRPs of all cells in the further set.

Optionally, UE may determine measured RSRPs and predicted RSRPs of partial cells (i.e., not all cells) in the set, and the partial cells may be determined based on NW’s configuration/indication, or determined based on certain pre-predefined criterion/threshold (e.g., top K (K≥1) cells in the set) .

For each cell in the set, UE may determine a RSRP difference between the measured RSRP and the predicted RSRP corresponding to the cell.

Furthermore, for each cell, UE may determine multiple (consecutive) RSRP differences corresponding to the cell over a period of time. Optionally, for each cell, UE may determine at least one statistical RSRP difference. The at least one statistical RSRP difference may comprise at least one of a mean value, maximum, minimum, median value, variance or standard deviation etc. of the multiple RSRP differences or a value representing data distribution of the multiple RSRP differences.

In some embodiments, for each cell in the set, UE may determine whether the cell is an inaccurate (or failure) predicted cell (optionally, determine whether the cell is an accurate predicted cell) based on the at least one RSRP difference or/and the at least one statistical RSRP difference corresponding to the cell.

Specifically, for a given cell, if at least one of the following conditions is fulfilled, (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

(Condition 4) In some embodiments, if the RSRP difference corresponding to the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the cell is an inaccurate predicted cell (or accurate predicted cell) .

(Condition 5) In some embodiments, if the measured RSRP of the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the cell is an inaccurate predicted cell (or accurate predicted cell) .

(Condition 6) In some embodiments, if the predicted RSRP of the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the cell is an inaccurate predicted cell (or accurate predicted cell) .

Additionally, in some embodiments, if the number of times (or probability of) at least one of Condition 4, 5 or 6 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the cell is an inaccurate predicted cell (or accurate predicted cell) .

Additionally, in some embodiments, if the at least one statistical RSRP difference corresponding to the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , the cell is an inaccurate predicted cell (or accurate predicted cell) .

Furthermore, (if there is at least one inaccurate predicted cell (or accurate predicted cell) ) , UE may determine whether a partial (or incomplete) model failure occurs, or/and a (full, complete or entire) model failure occurs based on the at least one inaccurate predicted cell (or accurate predicted cell) .

Specifically, if at least the following condition is fulfilled, (UE may assume) that a partial model failure occurs.

In some embodiments, if the number (or probability) of inaccurate predicted cells (or accurate predicted cells) is larger than or equal to a threshold (e.g., 1, or indicated by NW) (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume) that a partial model failure occurs.

In some embodiments, if the at least the following condition is fulfilled, (UE may assume) that a (full, complete or entire) model failure occurs.

In some embodiments, if the number (or probability) of inaccurate predicted cells (or accurate predicted cells) is larger than or equal to a threshold (e.g., equal to the total number of cells in the set, or indicated by NW) (over a period of time) , (UE may assume) that a (full, complete or entire) model failure occurs.

In some embodiments, (if there is at least one inaccurate predicted cell (or accurate predicted cell) ) , UE may report at least one of the following pieces of information (called as ‘second information’ ) to NW in at least one UL message/signaling.

● Indictor of inaccurate predicted cell (or accurate predicted cell) .

● RSRP difference corresponding to inaccurate predicted cell (or accurate predicted cell) .

● Statistical RSRP difference corresponding to inaccurate predicted cell (or accurate predicted cell) .

● The number of times at least one of Condition 4, 5 or 6 is fulfilled.

● Probability of at least one of Condition 4, 5 or 6 is fulfilled.

● The number of inaccurate predicted cells (or accurate predicted cells) .

● Probability of inaccurate predicted cells (or accurate predicted cells) .

● Information indicating whether a partial model failure occurs.

● Information indicating whether a model failure occurs.

In some embodiments, the at least one UL message/signaling may comprise at least one of UL RRC message, UL MAC CE, UCI. Method of reporting the second information is the same as that of reporting the first information, as discussed above.

In addition to the second information (e.g., the indictor of inaccurate predicted cell (or accurate predicted cell) ) , UE may report at least one of the following pieces of information (maybe together with the second information) :

● Indicator of set, e.g., the set comprising (or corresponding to) the inaccurate predicted cell (or accurate predicted cell) .

● Indicator of AI/ML model, e.g., the AI/ML model for determining predicted RSRP corresponding to the inaccurate predicted cell (or accurate predicted cell) .

● Indicator of predicted time instance, e.g., the predicted time instance corresponding to the inaccurate predicted cell (or accurate predicted cell) . Specifically, in case of temporal cell-level measurement prediction, UE may be configured with a set of predicted time instance, and each predicted time instance may be associated with a set of cells (these sets may be the same) .

● Indicator of zone, e.g., the zone associated with the inaccurate predicted cell (or accurate predicted cell) . Specifically, in case of inter-zone (spatial/temporal) cell-level measurement prediction, UE may be configured with a set of zones, and each zone may be associated with a set of cells.

Specifically, the above information may be used to indicate at least one of the following (but not limited to) : There is at least one inaccurate predicted cell (or accurate predicted cell) for the set, AI/ML model, predicted time instance or zone indicated by the above information, and/or whether a partial model failure or model failure occurs for the set, AI/ML model, predicted time instance or zone indicated by the above information.

Based on the inaccurate (or accurate) predicted cell determined and reported by UE, NW can configure reasonable cell related measurement resources and cell related prediction resources (or available predicted cells) . For example, NW may configure measurement resource for the inaccurate predicted cell reported by UE, so that UE can obtain the actual measured RSRP of the inaccurate predicted cell.

In addition, unnecessary entire model update/switching/fallback (to non-AI/ML based cell-level measurement prediction) can be avoided. Further, it is beneficial for reasonable model update, e.g., unnecessary overhead (i.e., data required for model update) and latency of model update can be reduced.

Example embodiments of classification based model and beam prediction will be discussed in the following, where the monitoring may be performed based on data distribution/prediction accuracy.

In operation, NW may configure for UE a set of beams and a further set of beams, which may be associated with at least one AI/ML model. Specifically, the set of beams may be used as predicted beams in model inference, i.e., the output (s) of the at least one AI/ML model may derive from the set of beams. The further set of beams may be used as measured beams in model inference, i.e., the input (s) of the at least one AI/ML model may derive from the measurements of the further set of beams.

Then, the UE may determine measured L1-RSRPs of all beams in the set and determine the top K1 beam (e.g., top-1 beam) in the set based on the measured L1-RSRPs. The top-1 beam may be called as ‘actual top-1 beam’ .

Further, UE may determine measured L1-RSRPs of all beams in the further set. In this case, UE may perform the following during performance monitoring.

In some embodiments, the monitoring may be performed based on data distribution. Specifically, it can be considered that the actual top-1 beam in the set (i.e., output data sample) corresponds to measured L1-RSRPs of all beams in the further set (i.e., input data sample) . During a period of time, one or multiple data samples may be obtained, thus, the same output sample (i.e., the same actual top-1 beam in the set) may correspond to one or multiple input data samples (i.e., measured L1-RSRPs of all beams in the further set) .

Since each beam in the set may be the actual top-1 beam, each beam in the set may correspond to one or multiple sets of measured L1-RSRPs of all beams in the further set (called as ‘set of measured L1-RSRPs’ for short) .

Then, UE may compare data distribution corresponding to the at least one set of measured L1-RSRPs with a reference data distribution (related to the at least one AI/ML model) . Based on the comparison, UE may determine at least one value (called as ‘data distribution difference’ for short) that is used to represent/reflect the difference between the data distribution corresponding to the at least one set of measured L1-RSRPs and the reference data distribution. For example, the value data distribution difference may be similarity or dissimilarity.

Alternatively, in some embodiment, monitoring may be performed based on beam prediction accuracy. Specifically, UE may determine the top K2 (K2 may be the same as or different from K1) beam (e.g., top-1 beam) in the set based on the measured L1-RSRPs of all beams in the further set and the at least one AI/ML model. This top-1 beam may be called as ‘predicted top-1 beam’ . It can be considered that the actual top-1 beam in the set corresponds to the actual top-1 beam in the set. Then, UE may determine a state that indicates whether the predicted top-1 beam is the same as the corresponding actual top-1 beam. For the convenience of description, ‘first state’ is used to indicate that the predicted top-1 beam is the same as the corresponding actual top-1 beam, and ‘second state’ is used to indicate that the predicted top-1 beam is different from the corresponding actual top-1 beam.

Since each beam in the set may be the actual or predicted top-1 beam, each beam in the set may correspond to one or multiple first or second states.

In a case that the monitoring may be performed based on data distribution, for each beam in the set, UE may determine whether the beam is an inaccurate predicted beam (or accurate predicted beam) based on the at least one data distribution difference and/or data distribution (s) corresponding to the at least one set of measured L1-RSRPs.

(Condition 7) In some embodiments, if the at least one data distribution difference corresponding to the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

(Condition 8) In some embodiments, if the value (s) representing the data distribution (s) corresponding to the at least one set of measured L1-RSRPs corresponding to the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

Additionally, in some embodiments, if the number of times (or probability of) at least one of Condition 7 or 8 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

In a case that the Monitoring based on beam prediction accuracy, for each beam in the set, UE may determine whether the beam is an inaccurate predicted beam (or accurate predicted beam) based on the at least one first or second states.

Specifically, for a given beam, if at least one of the following conditions is fulfilled, (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

(Condition 9) In some embodiments, if the number of (consecutive) occurrences of the second state (or first state) corresponding to the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

(Condition 10) Alternatively, or in addition, in some embodiments, if the probability of (consecutive) occurrences of the second state (or first state) corresponding to the beam is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

Additionally, in some embodiments, if the number of times (or probability of) at least one of Condition 9 or 10 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the beam is an inaccurate predicted beam (or accurate predicted beam) .

Furthermore, (if there is at least one inaccurate predicted beam (or accurate predicted beam) ) , UE may determine whether a partial (or incomplete) model failure occurs, or/and a (full, complete or entire) model failure occurs based on the at least one inaccurate predicted beam (or accurate predicted beam) . The determination method is the same as that discussed above.

In some embodiments, (if there is at least one inaccurate predicted beam (or accurate predicted beam) ) , UE may report at least one of the following pieces of information (called as ‘third information’ ) to NW in at least one UL message/signaling.

In a case of monitoring based on data distribution, the third information may comprise at least one of the following:

● Data distribution difference corresponding to inaccurate predicted beam (or accurate predicted beam) .

● Value representing data distribution (corresponding to the at least one set of measured L1-RSRPs) corresponding to inaccurate predicted beam (or accurate predicted beam) .

● Type (or format) of data distribution.

● The number of times at least one of Condition 7 or 8 is fulfilled.

● Probability of at least one of Condition 7 or 8 is fulfilled.

In a case of monitoring based on beam prediction accuracy, the third information may comprise at least one of the following:

● The number of (consecutive) occurrences of the second state (or first state) corresponding to inaccurate predicted beam (or accurate predicted beam) .

● The probability of (consecutive) occurrences of the second state (or first state) corresponding to inaccurate predicted beam (or accurate predicted beam) .

● The number of times at least one of Condition 9 or 10 is fulfilled.

● Probability of at least one of Condition 9 or 10 is fulfilled.

Additionally, regardless of monitoring based on data distribution or monitoring based on beam prediction accuracy, the third information may comprise at least one of the following:

● Indictor of inaccurate predicted beam (or accurate predicted beam) .

● The number of inaccurate predicted beams (or accurate predicted beams) .

● Probability of inaccurate predicted beams (or accurate predicted beams) .

● Information indicating whether a partial model failure occurs.

● Information indicating whether a model failure occurs.

In some embodiments, the at least one UL message/signaling may comprise at least one of UL RRC message, UL MAC CE, UCI.

Method of reporting the third information is the same as that of reporting the first information as discussed above.

In addition to the third information (e.g., the indictor of inaccurate predicted beam (or accurate predicted beam) ) , UE may report at least one of the following pieces of information (maybe together with the third information) : indicator of set (e.g., the set, the further set) , indicator of AI/ML model, indicator of predicted time instance, indicator of cell.

Based on the inaccurate (or accurate) predicted beam determined and reported by UE, when the predicted beam (e.g., top-1 beam) reported by UE is an inaccurate predicted beam, NW can promptly know this fact and make corresponding reasonable decision, e.g., fallback to non-AI/ML based beam management to determine the top-1 beam.

Example embodiments of classification based model and (target) cell prediction will be discussed in the following, where the monitoring may be performed based on data distribution/prediction accuracy.

In operation, NW may configure for UE a set of cells and a further set of cells (and optionally, each cell in the further set of cells may be associated with a set of beams (called as ‘beam set’ ) ) , which may be associated with at least one AI/ML model. Specifically, the set of cells may be used as predicted cells in model inference, i.e., the output (s) of the at least one AI/ML model may derive from the set of cells. The further set of cells (and the associated beam set) may be used as measured cells and/or beams in model inference, i.e., the input (s) of the at least one AI/ML model may derive from the measurements (e.g., L1-RSRP, RSRP) of the further set of cells and/or beams.

Then, the UE may determine measured RSRPs of all cells in the set and determine the top K1 cell (e.g., top-1 cell, or target cell) in the set based on the measured RSRPs. This target cell may be called as ‘actual target cell’ .

Further, UE may determine measured RSRPs of all cells in the further set, and/or measured L1-RSRPs of all beams of all beam sets associated with all cells in the further set, which is called as ‘measured L3/L1-RSRPs of the further set’ for short. In this case, UE may perform the following during performance monitoring.

In some embodiments, the monitoring may be performed based on data distribution. Specifically, it can be considered that the actual target cell in the set (i.e., output data sample) corresponds to the measured L3/L1-RSRPs of the further set (i.e., input data sample) . During a period of time, one or multiple data samples may be obtained, thus, the same output sample (i.e., the same actual target cell in the set) may correspond to one or multiple input data samples (i.e., measured L3/L1-RSRPs of the further set) .

Since each cell in the set may be the actual target cell, each cell in the set may correspond to one or multiple sets of measured L3/L1-RSRPs of the further set (called as ‘set of measured L3/L1-RSRPs’ for short) .

Then, UE may compare data distribution corresponding to the at least one set of measured L3/L1-RSRPs with a reference data distribution (related to the at least one AI/ML model) . Based on the comparison, UE may determine at least one data distribution difference between the data distribution corresponding to the at least one set of measured L3/L1-RSRPs and the reference data distribution.

Alternatively, in some embodiment, monitoring may be performed based on beam prediction accuracy. Specifically, UE may determine the top K2 (K2 may be the same as or different from K1) beam (e.g., top-1 beam) in the set based on the measured L3/L1-RSRPs of the further set and the at least one AI/ML model. This top-1 (or target) cell may be called as ‘predicted target cell’ . It can be considered that the actual target cell in the set corresponds to the actual target cell in the set. Then, UE may determine a state that indicates whether the predicted target cell is the same as the corresponding actual target cell. For the convenience of description, ‘first state’ is used to indicate that the predicted target cell is the same as the corresponding actual target cell, and ‘second state’ is used to indicate that the predicted target cell is different from the corresponding actual target cell.

Since each cell in the set may be the actual or predicted target cell, each cell in the set may correspond to one or multiple first or second states.

In a case that the monitoring may be performed based on data distribution, for each cell in the set, UE may determine whether the cell is an inaccurate predicted cell (or accurate predicted cell) based on the at least one data distribution difference and/or data distribution (s) corresponding to the at least one set of measured L3/L1-RSRPs.

(Condition 11) In some embodiments, if the at least one data distribution difference corresponding to the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

(Condition 12) In some embodiments, if the value (s) representing the data distribution (s) corresponding to the at least one set of measured L3/L1-RSRPs corresponding to the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

Additionally, in some embodiments, if the number of times (or probability of) at least one of Condition 11 or 12 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

In a case that the monitoring is performed based on beam prediction accuracy, for each cell in the set, UE may determine whether the cell is an inaccurate predicted cell (or accurate predicted cell) based on the at least one first or second states.

(Condition 13) In some embodiments, if the number of (consecutive) occurrences of the second state (or first state) corresponding to the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

(Condition 14) Alternatively, or in addition, in some embodiments, if the probability of (consecutive) occurrences of the second state (or first state) corresponding to the cell is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

Additionally, in some embodiments, if number of times (or probability of) at least one of Condition 13 or 14 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the cell is an inaccurate predicted cell (or accurate predicted cell) .

Furthermore, (if there is at least one inaccurate predicted cell (or accurate predicted cell) ) , UE may determine whether a partial (or incomplete) model failure occurs, or/and a (full, complete or entire) model failure occurs based on the at least one inaccurate predicted cell (or accurate predicted cell) . The determination method is the same as that mentioned in discussed above.

In some embodiments, (if there is at least one inaccurate predicted cell (or accurate predicted cell) ) , UE may report at least one of the following pieces of information (called as ‘fourth information’ ) to NW in at least one UL message/signaling.

In a case of monitoring based on data distribution, the fourth information may comprise at least one of the following:

Data distribution difference corresponding to inaccurate predicted cell (or accurate predicted cell) .

● Value representing data distribution (corresponding to the at least one set of measured L3/L1-RSRPs) corresponding to inaccurate predicted cell (or accurate predicted cell) .

● Type (or format) of data distribution.

● The number of times at least one of Condition 11 or 12 is fulfilled.

● Probability of at least one of Condition 11 or 12 is fulfilled.

In a case of monitoring based on beam prediction accuracy, the fourth information may comprise at least one of the following:

● The number of (consecutive) occurrences of the second state (or first state) corresponding to inaccurate predicted cell (or accurate predicted cell) .

● The probability of (consecutive) occurrences of the second state (or first state) corresponding to inaccurate predicted cell (or accurate predicted cell) .

● The number of times at least one of Condition 13 or 14 is fulfilled.

● Probability of at least one of Condition 13 or 14 is fulfilled.

Additionally, regardless of monitoring based on data distribution or monitoring based on beam prediction accuracy, the fourth information may comprise at least one of the following:

● Indictor of inaccurate predicted cell (or accurate predicted cell) .

● The number of inaccurate predicted cells (or accurate predicted cells) .

● Probability of inaccurate predicted cells (or accurate predicted cells) .

● Information indicating whether a partial model failure occurs.

● Information indicating whether a model failure occurs.

In addition to the fourth information (e.g., the indictor of inaccurate predicted cell (or accurate predicted cell) ) , UE may report at least one of the following pieces of information (maybe together with the fourth information) : indicator of set (e.g., the set) , indicator of AI/ML model, indicator of predicted time instance, indicator of zone.

Based on the inaccurate (or accurate) predicted cell determined and reported by UE, when the predicted target cell reported by UE is an inaccurate predicted cell, NW can promptly know this fact and make corresponding reasonable decision, e.g., fallback to non-AI/ML based mobility to determine the target cell.

In addition, unnecessary entire model update/switching/fallback can be avoided. Further, it is beneficial for reasonable model update, e.g., unnecessary overhead (i.e., data required for model update) and latency of model update can be reduced.

Example embodiments of classification based model and measurement event prediction will be discussed in the following, where the monitoring may be performed based on data distribution/prediction accuracy.

As a generally rule, in case of measurement event (e.g., Event A1～A6) prediction (using classification based model) , all procedures, all information and all methods for determining and reporting the information discussed with reference to example embodiments of classification based model and (target) cell prediction is applicable to this case, just need to replace ‘cell’ with ‘measurement event’ .

In operation, NW may configure for UE a set of measurement events and a further set of measurement events (and optionally, each measurement event in the further set of measurement events may be associated with a set of beams (called as ‘beam set’ ) ) , which may be associated with at least one AI/ML model. Specifically, the set of measurement events may be used as predicted measurement events in model inference, i.e., the output (s) of the at least one AI/ML model may derive from the set of measurement events. The further set of measurement events (and the associated beam set) may be used as measured measurement events and/or beams in model inference, i.e., the input (s) of the at least one AI/ML model may derive from the measurements (e.g., L1-RSRP, RSRP) of the further set of measurement events and/or beams.

Then, the UE may determine measured RSRPs of all measurement events in the set and determine the top K1 measurement event (e.g., top-1 measurement event, or target measurement event) in the set based on the measured RSRPs. This target measurement event may be called as ‘actual target measurement event’ .

Further, UE may determine measured RSRPs of all measurement events in the further set, and/or measured L1-RSRPs of all beams of all beam sets associated with all measurement events in the further set, which is called as ‘measured L3/L1-RSRPs of the further set’ for short. In this case, UE may perform the following during performance monitoring.

In some embodiments, the monitoring may be performed based on data distribution. Specifically, it can be considered that the actual target measurement event in the set (i.e., output data sample) corresponds to the measured L3/L1-RSRPs of the further set (i.e., input data sample) . During a period of time, one or multiple data samples may be obtained, thus, the same output sample (i.e., the same actual target measurement event in the set) may correspond to one or multiple input data samples (i.e., measured L3/L1-RSRPs of the further set) .

Since each measurement event in the set may be the actual target measurement event, each measurement event in the set may correspond to one or multiple sets of measured L3/L1-RSRPs of the further set (called as ‘set of measured L3/L1-RSRPs’ for short) .

Alternatively, in some embodiment, monitoring may be performed based on beam prediction accuracy. Specifically, UE may determine the top K2 (K2 may be the same as or different from K1) beam (e.g., top-1 beam) in the set based on the measured L3/L1-RSRPs of the further set and the at least one AI/ML model. This top-1 (or target) measurement event may be called as ‘predicted target measurement event’ . It can be considered that the actual target measurement event in the set corresponds to the actual target measurement event in the set. Then, UE may determine a state that indicates whether the predicted target measurement event is the same as the corresponding actual target measurement event. For the convenience of description, ‘first state’ is used to indicate that the predicted target measurement event is the same as the corresponding actual target measurement event, and ‘second state’ is used to indicate that the predicted target measurement event is different from the corresponding actual target measurement event.

Since each measurement event in the set may be the actual or predicted target measurement event, each measurement event in the set may correspond to one or multiple first or second states.

In a case that the monitoring may be performed based on data distribution, for each measurement event in the set, UE may determine whether the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) based on the at least one data distribution difference and/or data distribution (s) corresponding to the at least one set of measured L3/L1-RSRPs.

(Condition 15) In some embodiments, if the at least one data distribution difference corresponding to the measurement event is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) .

(Condition 16) In some embodiments, if the value (s) representing the data distribution (s) corresponding to the at least one set of measured L3/L1-RSRPs corresponding to the measurement event is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) .

Additionally, in some embodiments, if the number of times (or probability of) at least one of Condition 15 or 16 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) .

In a case that the monitoring is performed based on beam prediction accuracy, for each measurement event in the set, UE may determine whether the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) based on the at least one first or second states.

(Condition 17) In some embodiments, if the number of (consecutive) occurrences of the second state (or first state) corresponding to the measurement event is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) .

(Condition 18) Alternatively, or in addition, in some embodiments, if the probability of (consecutive) occurrences of the second state (or first state) corresponding to the measurement event is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) .

Additionally, in some embodiments, if number of times (or probability of) at least one of Condition 17 or 18 is fulfilled is larger than or equal to a threshold (over a period of time) , or/and it is smaller than or equal to a threshold (over a period of time) , (UE may assume that) the measurement event is an inaccurate predicted measurement event (or accurate predicted measurement event) .

Furthermore, (if there is at least one inaccurate predicted measurement event (or accurate predicted measurement event) ) , UE may determine whether a partial (or incomplete) model failure occurs, or/and a (full, complete or entire) model failure occurs based on the at least one inaccurate predicted measurement event (or accurate predicted measurement event) . The determination method is the same as that mentioned in discussed above.

In some embodiments, (if there is at least one inaccurate predicted measurement event (or accurate predicted measurement event) ) , UE may report at least one of the following pieces of information (called as ‘fifth information’ ) to NW in at least one UL message/signaling.

In a case of monitoring based on data distribution, the fifth information may comprise at least one of the following:

Data distribution difference corresponding to inaccurate predicted measurement event (or accurate predicted measurement event) .

● Value representing data distribution (corresponding to the at least one set of measured L3/L1-RSRPs) corresponding to inaccurate predicted measurement event (or accurate predicted measurement event) .

● Type (or format) of data distribution.

● The number of times at least one of Condition 15 or 16 is fulfilled.

● Probability of at least one of Condition 15 or 16 is fulfilled.

In a case of monitoring based on beam prediction accuracy, the fifth information may comprise at least one of the following:

● The number of (consecutive) occurrences of the second state (or first state) corresponding to inaccurate predicted measurement event (or accurate predicted measurement event) .

● The probability of (consecutive) occurrences of the second state (or first state) corresponding to inaccurate predicted measurement event (or accurate predicted measurement event) .

● The number of times at least one of Condition 17 or 18 is fulfilled.

● Probability of at least one of Condition 17 or 18 is fulfilled.

Additionally, regardless of monitoring based on data distribution or monitoring based on beam prediction accuracy, the fifth information may comprise at least one of the following:

● Indictor of inaccurate predicted measurement event (or accurate predicted measurement event) .

● The number of inaccurate predicted measurement events (or accurate predicted measurement events) .

● Probability of inaccurate predicted measurement events (or accurate predicted measurement events) .

● Information indicating whether a partial model failure occurs.

● Information indicating whether a model failure occurs.

In addition to the fifth information (e.g., the indictor of inaccurate predicted measurement event (or accurate predicted measurement event) ) , UE may report at least one of the following pieces of information (maybe together with the fifth information) : indicator of set (e.g., the set) , indicator of AI/ML model, indicator of predicted time instance, indicator of zone.

Based on the inaccurate (or accurate) predicted measurement event determined and reported by UE, when the predicted target measurement event reported by UE is an inaccurate predicted measurement event, NW can promptly know this fact and make corresponding reasonable decision, e.g., fallback to non-AI/ML based mobility to determine the target measurement event.

Example methods

FIG. 3 illustrates a flowchart of a communication method 300 implemented at a first device in accordance with some embodiments of the present disclosure. For the purpose of discussion, the method 300 will be described from the perspective of the first device 110 in FIG. 1.

At block 310, the first device determines inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event.

At block 320, the first device transmits at least one message to a second device, the at least one message indicating at least one of the following: the inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information.

In some example embodiments, the first device may determine a first set of measurement results of a first set of elements, wherein the first set of elements is a first set of beams or a first set of cells; determine, by the at least one ML model, a first set of prediction results of the first set of elements; and for each element in the first set, determine whether the element is an inaccurate predicted element based on at least one of the following: at least one first difference, wherein each first difference is determined based on a measurement result of the element and a prediction result of the element, or at least one statistical value of first differences.

In some example embodiments, the statistical value is one of the following: a mean value, a maximum value, a minimum value, a median value, a variance, or a standard deviation.

In some example embodiments, the first set of elements is a set of beams associated with at least one output of the at least one ML model, the first set of elements is a set of cells associated with at least one output of the at least one ML model, the first set of elements is configured by the second device, the first set of elements is determined based on a pre-predefined criterion, or the first set of elements is determined based on a pre-predefined threshold.

In some example embodiments, the element is determined to be an inaccurate predicted element if at least one of the following: a first difference between a measurement result of the element and a prediction result of the element is equal to or larger than a first threshold, a number of times that the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold is equal to or larger than a second threshold, the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold over a first period of time, a statistical value of first differences of the element is equal to or larger than a third threshold, a number of times that the statistical value of first differences of the element is equal to or larger than a third threshold is equal to or larger than a fourth threshold, or the statistical value of first differences of the element is equal to or larger than a third threshold over a second period of time.

In some example embodiments, the element is determined to be an inaccurate predicted element if at least one of the following: a measurement result of the element is equal to or smaller than a fifth threshold, a number of times that the measurement result of the beam or cell is equal to or smaller than a fifth threshold is equal to or larger than a sixth threshold, the measurement result of the beam or cell is equal to or smaller than a fifth threshold over a third period of time, a prediction result of the beam or cell is equal to or larger than a seventh threshold, a number of times that the prediction result of the beam or cell is equal to or larger than a seventh threshold is equal to or larger than an eighth threshold, the prediction result of the beam or cell is equal to or larger than a seventh threshold over a fourth period of time.

In some example embodiments, for an element in a set of elements, the first device may determine whether the element is an inaccurate predicted element based on the following: at least one comparation result associated with the element, each comparation result indicating whether a first order of the element in a set of measurement results is consistent with a second order of the element in a set of prediction results, wherein the element is one of the following: a beam, a cell, or an event.

In some example embodiments, the element is determined to be an inaccurate element if at least one of the following: the comparation result indicates that the first order of the element is inconsistent with the second order of the element, a number of times that the comparation result indicates that the first order of the element is inconsistent with the second order of the element is equal to or larger than a first threshold, or the comparation result indicates that the first quality order of the element is inconsistent with the second quality order over a first period of time.

In some example embodiments, for an element in a set of elements, the first device may determine whether the element is an inaccurate predicted element based on at least one of the following: at least one first data distribution associated with the element; or at least one second difference associated with the element, wherein each second difference is determined based on a first data distribution associated with the element and a reference data distribution, wherein the element is one of the following: a beam, a cell, or an event.

In some example embodiments, the element is determined to be an inaccurate predicted element if at least one of the following: a value representing the first data distribution of the element is larger than or equal to a threshold a first threshold, a number of times that the value representing the first data distribution of the element is larger than or equal to a threshold a first threshold is equal to or larger than a second threshold, the value representing the first data distribution of the element is larger than or equal to a first threshold over a first period, the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third threshold, a number of times that the second difference is equal to or larger than a third threshold is equal to or larger than a fourth threshold, the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third fourth over a third period of time.

In some example embodiments, the first device may determine a partial model failure occurs if at least one of the following: a number of detected inaccurate predicted elements is equal to or smaller than a threshold, a number of times of detecting an inaccurate predicted element is equal to or smaller than a threshold; or determine a full model failure occurs if at least one of the following: a number of detected inaccurate predicted elements is equal to or larger than a threshold, a number of times of detecting an inaccurate predicted element is equal to or larger than a threshold, wherein the element is one of the following: a beam, a cell, or an event.

In some example embodiments, the inaccurate prediction information comprises at least one of the following: at least one indicator of the at least one inaccurate predicted beam, at least one indicator of the at least one inaccurate predicted cell, or at least one indicator of the at least one inaccurate predicted event.

In some example embodiments, the element is one of the following: a beam, a cell, or an event, and wherein the at least one message indicates at least one of the following: a number of detected inaccurate predicted elements, a number of times of detecting an inaccurate predicted element, a statistical value of first differences associated with an inaccurate predicted element, a first difference between a measurement result of an inaccurate predicted element and a prediction result of the inaccurate predicted element, a second difference between a first data distribution associated with an inaccurate predicted element and a reference data distribution, a value representing the second difference, a type of the first data distribution associated with an inaccurate predicted element, a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is inconsistent with a second quality order of the element determined based on a set of prediction results, a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is consistent with a second quality order of the element determined based on a set of prediction results, an indication used for requesting a re-configuration of a set of beams or cells associated with an output of the at least one ML model, an indication used for requesting a re-configuration of a set of cells associated with an output of the at least one ML model, an identity of a set of beams associated with the inaccurate predicted element, an identity of a set of cells associated with the inaccurate predicted element, an indicator of a predicted time instance associated with the inaccurate predicted element, an indicator of a cell associated with the inaccurate predicted element, an indicator of a zone associated with the inaccurate predicted element, or an identity of the at least one ML model associated with the inaccurate predicted element.

In some example embodiments, the at least one message comprises at least one of the following: radio resource control (RRC) signalling, a medium access control (MAC) control element (CE) or uplink control information (UCI) .

In some example embodiments, the at least one message comprises at least one of a measurement report, a user equipment (UE) assistance information (UAI) , or a UE capability information.

In some example embodiments, the at least one message is transmitted on at least one of a physical uplink control channel (PUCCH) resource or a physical uplink shared channel (PUSCH) resource associated with a predefined scheduling request (SR) .

In some example embodiments, the first device is a terminal device, and the second device is a network device.

FIG. 4 illustrates a flowchart of a communication method 400 implemented at a second device in accordance with some embodiments of the present disclosure. For the purpose of discussion, the method 400 will be described from the perspective of the second device in FIG. 1.

At block 410, the second device receive, at least one message from a first device, the at least one message indicating at least one of the following: inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information. The inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event.

Example Apparatus and Devices

FIG. 5 is a simplified block diagram of a device 500 that is suitable for implementing embodiments of the present disclosure. The device 500 can be considered as a further example implementation of any of the devices as shown in FIG. 1. Accordingly, the device 500 can be implemented at or as at least a part of the first device 110 or the second device 120.

As shown, the device 500 includes a processor 510, a memory 520 coupled to the processor 510, a suitable transceiver 540 coupled to the processor 510, and a communication interface coupled to the transceiver 540. The memory 520 stores at least a part of a program 530. The transceiver 540 may be for bidirectional communications or a unidirectional communication based on requirements. The transceiver 540 may include at least one of a transmitter 542 and a receiver 544. The transmitter 542 and the receiver 544 may be functional modules or physical entities. The transceiver 540 has at least one antenna to facilitate communication, though in practice an Access Node mentioned in this application may have several ones. The communication interface may represent any interface that is necessary for communication with other network elements, such as X2/Xn interface for bidirectional communications between eNBs/gNBs, S1/NG interface for communication between a Mobility Management Entity (MME) /Access and Mobility Management Function (AMF) /SGW/UPF and the eNB/gNB, Un interface for communication between the eNB/gNB and a relay node (RN) , or Uu interface for communication between the eNB/gNB and a terminal device.

The program 530 is assumed to include program instructions that, when executed by the associated processor 510, enable the device 500 to operate in accordance with the embodiments of the present disclosure, as discussed herein with reference to FIGS. 1 to 5. The embodiments herein may be implemented by computer software executable by the processor 510 of the device 500, or by hardware, or by a combination of software and hardware. The processor 510 may be configured to implement various embodiments of the present disclosure. Furthermore, a combination of the processor 510 and memory 520 may form processing means 550 adapted to implement various embodiments of the present disclosure.

The memory 520 may be of any type suitable to the local technical network and may be implemented using any suitable data storage technology, such as a non-transitory computer readable storage medium, semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory, as non-limiting examples. While only one memory 520 is shown in the device 500, there may be several physically distinct memory modules in the device 500. The processor 510 may be of any type suitable to the local technical network, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 500 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

According to embodiments of the present disclosure, a first device comprising a circuitry is provided. The circuitry is configured to: determine inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event; and transmit at least one message to a second device, the at least one message indicating at least one of the following: the inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information. According to embodiments of the present disclosure, the circuitry may be configured to perform any method implemented by the first device as discussed above.

According to embodiments of the present disclosure, a second device comprising a circuitry is provided. The circuitry is configured to: receive, at least one message from a first device, the at least one message indicating at least one of the following: inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information, wherein the inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event. According to embodiments of the present disclosure, the circuitry may be configured to perform any method implemented by the second device as discussed above.

The term “circuitry” used herein may refer to hardware circuits and/or combinations of hardware circuits and software. For example, the circuitry may be a combination of analog and/or digital hardware circuits with software/firmware. As a further example, the circuitry may be any portions of hardware processors with software including digital signal processor (s) , software, and memory (ies) that work together to cause an apparatus, such as a terminal device or a network device, to perform various functions. In a still further example, the circuitry may be hardware circuits and or processors, such as a microprocessor or a portion of a microprocessor, that requires software/firmware for operation, but the software may not be present when it is not needed for operation. As used herein, the term circuitry also covers an implementation of merely a hardware circuit or processor (s) or a portion of a hardware circuit or processor (s) and its (or their) accompanying software and/or firmware.

According to embodiments of the present disclosure, a first apparatus is provided. The first apparatus comprises means for determining inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event; and means for transmitting at least one message to a second device, the at least one message indicating at least one of the following: means for the inaccurate prediction information, means for a partial model failure determined based on the inaccurate prediction information, or means for a full model failure determined based on the inaccurate prediction information. In some embodiments, the first apparatus may comprise means for performing the respective operations of the method 300. In some example embodiments, the first apparatus may further comprise means for performing other operations in some example embodiments of the method 300. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

According to embodiments of the present disclosure, a second apparatus is provided. The second apparatus comprises means for receiving, at least one message from a first device, the at least one message indicating at least one of the following: means for inaccurate prediction information, means for a partial model failure determined based on the inaccurate prediction information, or means for a full model failure determined based on the inaccurate prediction information, wherein the inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event. In some embodiments, the second apparatus may comprise means for performing the respective operations of the method 400. In some example embodiments, the second apparatus may further comprise means for performing other operations in some example embodiments of the method 400. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

In summary, embodiments of the present disclosure provide the following aspects.

In an aspect, it is proposed a first device comprising: a processor configured to cause the first device to: determine inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event; and transmit at least one message to a second device, the at least one message indicating at least one of the following: the inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information.

In some embodiments, the processor is further configured to cause the first device to: determine a first set of measurement results of a first set of elements, wherein the first set of elements is a first set of beams or a first set of cells; determine, by the at least one ML model, a first set of prediction results of the first set of elements; and for each element in the first set, determine whether the element is an inaccurate predicted element based on at least one of the following: at least one first difference, wherein each first difference is determined based on a measurement result of the element and a prediction result of the element, or at least one statistical value of first differences.

In some embodiments, the statistical value is one of the following: a mean value, a maximum value, a minimum value, a median value, a variance, or a standard deviation.

In some embodiments, the first set of elements is a set of beams associated with at least one output of the at least one ML model, the first set of elements is a set of cells associated with at least one output of the at least one ML model, the first set of elements is configured by the second device, the first set of elements is determined based on a pre-predefined criterion, or the first set of elements is determined based on a pre-predefined threshold.

In some embodiments, the element is determined to be an inaccurate predicted element if at least one of the following: a first difference between a measurement result of the element and a prediction result of the element is equal to or larger than a first threshold, a number of times that the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold is equal to or larger than a second threshold, the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold over a first period of time, a statistical value of first differences of the element is equal to or larger than a third threshold, a number of times that the statistical value of first differences of the element is equal to or larger than a third threshold is equal to or larger than a fourth threshold, or the statistical value of first differences of the element is equal to or larger than a third threshold over a second period of time.

In some embodiments, the element is determined to be an inaccurate predicted element if at least one of the following: a measurement result of the element is equal to or smaller than a fifth threshold, a number of times that the measurement result of the beam or cell is equal to or smaller than a fifth threshold is equal to or larger than a sixth threshold, the measurement result of the beam or cell is equal to or smaller than a fifth threshold over a third period of time, a prediction result of the beam or cell is equal to or larger than a seventh threshold, a number of times that the prediction result of the beam or cell is equal to or larger than a seventh threshold is equal to or larger than an eighth threshold, the prediction result of the beam or cell is equal to or larger than a seventh threshold over a fourth period of time.

In some embodiments, for an element in a set of elements, the first device may determine whether the element is an inaccurate predicted element based on the following: at least one comparation result associated with the element, each comparation result indicating whether a first order of the element in a set of measurement results is consistent with a second order of the element in a set of prediction results, wherein the element is one of the following: a beam, a cell, or an event.

In some embodiments, the element is determined to be an inaccurate element if at least one of the following: the comparation result indicates that the first order of the element is inconsistent with the second order of the element, a number of times that the comparation result indicates that the first order of the element is inconsistent with the second order of the element is equal to or larger than a first threshold, or the comparation result indicates that the first quality order of the element is inconsistent with the second quality order over a first period of time.

In some embodiments, for an element in a set of elements, the first device may determine whether the element is an inaccurate predicted element based on at least one of the following: at least one first data distribution associated with the element; or at least one second difference associated with the element, wherein each second difference is determined based on a first data distribution associated with the element and a reference data distribution, wherein the element is one of the following: a beam, a cell, or an event.

In some embodiments, the element is determined to be an inaccurate predicted element if at least one of the following: a value representing the first data distribution of the element is larger than or equal to a threshold a first threshold, a number of times that the value representing the first data distribution of the element is larger than or equal to a threshold a first threshold is equal to or larger than a second threshold, the value representing the first data distribution of the element is larger than or equal to a first threshold over a first period, the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third threshold, a number of times that the second difference is equal to or larger than a third threshold is equal to or larger than a fourth threshold, the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third fourth over a third period of time.

In some embodiments, the first device may determine a partial model failure occurs if at least one of the following: a number of detected inaccurate predicted elements is equal to or smaller than a threshold, a number of times of detecting an inaccurate predicted element is equal to or smaller than a threshold; or determine a full model failure occurs if at least one of the following: a number of detected inaccurate predicted elements is equal to or larger than a threshold, a number of times of detecting an inaccurate predicted element is equal to or larger than a threshold, wherein the element is one of the following: a beam, a cell, or an event.

In some embodiments, the inaccurate prediction information comprises at least one of the following: at least one indicator of the at least one inaccurate predicted beam, at least one indicator of the at least one inaccurate predicted cell, or at least one indicator of the at least one inaccurate predicted event.

In some embodiments, the element is one of the following: a beam, a cell, or an event, and wherein the at least one message indicates at least one of the following: a number of detected inaccurate predicted elements, a number of times of detecting an inaccurate predicted element, a statistical value of first differences associated with an inaccurate predicted element, a first difference between a measurement result of an inaccurate predicted element and a prediction result of the inaccurate predicted element, a second difference between a first data distribution associated with an inaccurate predicted element and a reference data distribution, a value representing the second difference, a type of the first data distribution associated with an inaccurate predicted element, a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is inconsistent with a second quality order of the element determined based on a set of prediction results, a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is consistent with a second quality order of the element determined based on a set of prediction results, an indication used for requesting a re-configuration of a set of beams or cells associated with an output of the at least one ML model, an indication used for requesting a re-configuration of a set of cells associated with an output of the at least one ML model, an identity of a set of beams associated with the inaccurate predicted element, an identity of a set of cells associated with the inaccurate predicted element, an indicator of a predicted time instance associated with the inaccurate predicted element, an indicator of a cell associated with the inaccurate predicted element, an indicator of a zone associated with the inaccurate predicted element, or an identity of the at least one ML model associated with the inaccurate predicted element.

In some embodiments, the at least one message comprises at least one of the following: radio resource control (RRC) signalling, a medium access control (MAC) control element (CE) or uplink control information (UCI) .

In some embodiments, the at least one message comprises at least one of a measurement report, a user equipment (UE) assistance information (UAI) , or a UE capability information.

In some embodiments, the at least one message is transmitted on at least one of a physical uplink control channel (PUCCH) resource or a physical uplink shared channel (PUSCH) resource associated with a predefined scheduling request (SR) .

In some embodiments, the first device is a terminal device, and the second device is a network device.

In an aspect, it is proposed a second device comprising: a processor configured to cause the second device to: receive, at least one message from a first device, the at least one message indicating at least one of the following: inaccurate prediction information, a partial model failure determined based on the inaccurate prediction information, or a full model failure determined based on the inaccurate prediction information, wherein the inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following: at least one inaccurate predicted beam, at least one inaccurate predicted cell, or at least one inaccurate predicted event.

In an aspect, a first device comprises: at least one processor; and at least one memory coupled to the at least one processor and storing instructions thereon, the instructions, when executed by the at least one processor, causing the device to perform the method implemented by the first device discussed above.

In an aspect, a second device comprises: at least one processor; and at least one memory coupled to the at least one processor and storing instructions thereon, the instructions, when executed by the at least one processor, causing the device to perform the method implemented by the second device discussed above.

In an aspect, a computer readable medium having instructions stored thereon, the instructions, when executed on at least one processor, causing the at least one processor to perform the method implemented by the first device discussed above.

In an aspect, a computer readable medium having instructions stored thereon, the instructions, when executed on at least one processor, causing the at least one processor to perform the method implemented by the second device discussed above.

In an aspect, a computer program comprising instructions, the instructions, when executed on at least one processor, causing the at least one processor to perform the method implemented by the first device discussed above.

In an aspect, a computer program comprising instructions, the instructions, when executed on at least one processor, causing the at least one processor to perform the method implemented by the second device discussed above.

Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representation, it will be appreciated that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer readable storage medium. The computer program product includes computer-executable instructions, such as those included in program modules, being executed in a device on a target real or virtual processor, to carry out the process or method as described above with reference to FIGS. 1 to 5. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowcharts and/or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

The above program code may be embodied on a machine readable medium, which may be any tangible medium that may contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. A machine readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM) , a read-only memory (ROM) , an erasable programmable read-only memory (EPROM or Flash memory) , an optical fiber, a portable compact disc read-only memory (CD-ROM) , an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable sub-combination.

Although the present disclosure has been described in language specific to structural features and/or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

A first device comprising:

a processor configured to cause the first device to:

determine inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following:

at least one inaccurate predicted beam,

at least one inaccurate predicted cell, or

at least one inaccurate predicted event; and

transmit at least one message to a second device, the at least one message indicating at least one of the following:

the inaccurate prediction information,

a partial model failure determined based on the inaccurate prediction information, or

a full model failure determined based on the inaccurate prediction information.
The first device of claim 1, wherein the processor is further configured to cause the first device to:

determine a first set of measurement results of a first set of elements, wherein the first set of elements is a first set of beams or a first set of cells;

determine, by the at least one ML model, a first set of prediction results of the first set of elements; and

for each element in the first set, determine whether the element is an inaccurate predicted element based on at least one of the following:

at least one first difference, wherein each first difference is determined based on a measurement result of the element and a prediction result of the element, or

at least one statistical value of first differences.
The first device of claim 2, wherein the statistical value is one of the following: a mean value, a maximum value, a minimum value, a median value, a variance, or a standard deviation.
The first device of claim 2, wherein the element is determined to be an inaccurate predicted element if at least one of the following:

a first difference between a measurement result of the element and a prediction result of the element is equal to or larger than a first threshold,

a number of times that the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold is equal to or larger than a second threshold,

the first difference between the measurement result of the element and the prediction result of the element is equal to or larger than a first threshold over a first period of time,

a statistical value of first differences of the element is equal to or larger than a third threshold,

a number of times that the statistical value of first differences of the element is equal to or larger than a third threshold is equal to or larger than a fourth threshold, or

the statistical value of first differences of the element is equal to or larger than a third threshold over a second period of time.
The first device of claim 1, wherein the processor is further configured to cause the first device to:

for an element in a set of elements, determine whether the element is an inaccurate predicted element based on the following: at least one comparation result associated with the element, each comparation result indicating whether a first order of the element in a set of measurement results is consistent with a second order of the element in a set of prediction results,

wherein the element is one of the following: a beam, a cell, or an event.
the first device of claim 5, wherein the element is determined to be an inaccurate element if at least one of the following:

the comparation result indicates that the first order of the element is inconsistent with the second order of the element,

a number of times that the comparation result indicates that the first order of the element is inconsistent with the second order of the element is equal to or larger than a first threshold, or

the comparation result indicates that the first quality order of the element is inconsistent with the second quality order over a first period of time.
The first device of claim 1, wherein the element is one of the following: a beam, a cell, or an event and wherein the processor is further configured to cause the first device to:

for an element in a set of elements, determine whether the element is an inaccurate predicted element based on at least one of the following:

at least one first data distribution associated with the element; or

at least one second difference associated with the element, wherein each second difference is determined based on a first data distribution associated with the element and a reference data distribution.
The first device of claim 7, wherein the element is determined to be an inaccurate predicted element if at least one of the following:

a value representing the first data distribution of the element is larger than or equal to a threshold a first threshold,

a number of times that the value representing the first data distribution of the element is larger than or equal to a threshold a first threshold is equal to or larger than a second threshold,

the value representing the first data distribution of the element is larger than or equal to a first threshold over a first period,

the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third threshold,

a number of times that the second difference is equal to or larger than a third threshold is equal to or larger than a fourth threshold, or

the second difference determined based on the first data distribution and the reference data distribution is equal to or larger than a third fourth over a third period of time.
The first device of claim 1, wherein the element is one of the following: a beam, a cell, or an event and wherein the processor is further configured to cause the first device to:

determine a partial model failure occurs if at least one of the following:

a number of detected inaccurate predicted elements is equal to or smaller than a threshold, or

a number of times of detecting an inaccurate predicted element is equal to or smaller than a threshold; or

determine a full model failure occurs if at least one of the following:

a number of detected inaccurate predicted elements is equal to or larger than a threshold, or

a number of times of detecting an inaccurate predicted element is equal to or larger than a threshold.
The first device of claim 1, wherein the inaccurate prediction information comprises at least one of the following:

at least one indicator of the at least one inaccurate predicted beam,

at least one indicator of the at least one inaccurate predicted cell, or

at least one indicator of the at least one inaccurate predicted event.
The first device of claim 1, wherein the element is one of the following: a beam, a cell, or an event, and wherein the at least one message indicates at least one of the following:

a number of detected inaccurate predicted elements,

a number of times of detecting an inaccurate predicted element,

a statistical value of first differences associated with an inaccurate predicted element,

a first difference between a measurement result of an inaccurate predicted element and a prediction result of the inaccurate predicted element,

a second difference between a first data distribution associated with an inaccurate predicted element and a reference data distribution,

a value representing the second difference,

a type of the first data distribution associated with an inaccurate predicted element,

a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is inconsistent with a second quality order of the element determined based on a set of prediction results,

a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is consistent with a second quality order of the element determined based on a set of prediction results,

an indication used for requesting a re-configuration of a set of beams or cells associated with an output of the at least one ML model,

an indication used for requesting a re-configuration of a set of cells associated with an output of the at least one ML model,

an identity of a set of beams associated with the inaccurate predicted element,

an identity of a set of cells associated with the inaccurate predicted element,

an indicator of a predicted time instance associated with the inaccurate predicted element,

an indicator of a cell associated with the inaccurate predicted element,

an indicator of a zone associated with the inaccurate predicted element, or

an identity of the at least one ML model associated with the inaccurate predicted element.
The first device of claim 1, wherein the at least one message comprises at least one of the following: radio resource control (RRC) signalling, a medium access control (MAC) control element (CE) or uplink control information (UCI) .
The first device of claim 1, wherein the at least one message comprises at least one of a measurement report, a user equipment (UE) assistance information (UAI) , or a UE capability information.
The first device of claim 1, wherein the at least one message is transmitted on at least one of a physical uplink control channel (PUCCH) resource or a physical uplink shared channel (PUSCH) resource associated with a predefined scheduling request (SR) .
The first device of claim 1, wherein the first device is a terminal device, and the second device is a network device.
A second device comprising:

a processor configured to cause the second device to:

receive, at least one message from a first device, the at least one message indicating at least one of the following:

inaccurate prediction information,

a partial model failure determined based on the inaccurate prediction information, or

a full model failure determined based on the inaccurate prediction information,

wherein the inaccurate prediction information associated with at least one machine learning (ML) model at the first device, the inaccurate prediction information related to at least of the following:

at least one inaccurate predicted beam,

at least one inaccurate predicted cell, or

at least one inaccurate predicted event.
The second device of claim 16, wherein the inaccurate prediction information comprises at least one of the following:

at least one indicator of the at least one inaccurate predicted beam,

at least one indicator of the at least one inaccurate predicted cell, or

at least one indicator of the at least one inaccurate predicted event.
The second device of claim 16, wherein the element is one of the following: a beam, a cell, or an event, and wherein the at least one message indicates at least one of the following:

a number of detected inaccurate predicted elements,

a number of times of detecting an inaccurate predicted element,

a statistical value of first differences associated with an inaccurate predicted element,

a first difference between a measurement result of an inaccurate predicted element and a prediction result of the inaccurate predicted element,

a second difference between a first data distribution associated with an inaccurate predicted element and a reference data distribution,

a value representing the second difference,

a type of the first data distribution associated with an inaccurate predicted element,

a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is inconsistent with a second quality order of the element determined based on a set of prediction results,

a number of time that a comparation result indicates a first quality order of an element determined based on a set of measurement results is consistent with a second quality order of the element determined based on a set of prediction results,

an indication used for requesting a re-configuration of a set of beams or cells associated with an output of the at least one ML model,

an indication used for requesting a re-configuration of a set of cells associated with an output of the at least one ML model,

an identity of a set of beams associated with the inaccurate predicted element,

an identity of a set of cells associated with the inaccurate predicted element,

an indicator of a predicted time instance associated with the inaccurate predicted element,

an indicator of a cell associated with the inaccurate predicted element,

an indicator of a zone associated with the inaccurate predicted element, or

an identity of the at least one ML model associated with the inaccurate predicted element.
The second device of claim 16, wherein the at least one message comprises at least one of the following: radio resource control (RRC) signalling, a medium access control (MAC) control element (CE) or uplink control information (UCI) .
The second device of claim 16, wherein the at least one message comprises at least one of a measurement report, a user equipment (UE) assistance information (UAI) , or a UE capability information.
The second device of claim 16, wherein the at least one message is transmitted on at least one of a physical uplink control channel (PUCCH) resource or a physical uplink shared channel (PUSCH) resource associated with a predefined scheduling request (SR) .
The second device of claim 16, wherein the first device is a terminal device, and the second device is a network device.