WO2024035086A1

WO2024035086A1 - Ue measurement capability indication for ai/ml dataset construction

Info

Publication number: WO2024035086A1
Application number: PCT/KR2023/011699
Authority: WO
Inventors: Mythri Hunukumbure; Chadi KHIRALLAH
Original assignee: Samsung Electronics Co., Ltd.
Priority date: 2022-08-10
Filing date: 2023-08-08
Publication date: 2024-02-15
Also published as: GB2624844A; GB202211666D0

Abstract

A method of a terminal for obtaining measurement data for AI/ML model training in a 5G NR or beyond communications system comprising one or more network entities and one or more terminals, the method comprising: transmitting, from the terminal to a network entity, information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training; measuring, at the terminal, one or more values associated with the data for AI/ML model training; and transmitting, from the terminal to the network entity, the measured values.

Description

UE MEASUREMENT CAPABILITY INDICATION FOR AI/ML DATASET CONSTRUCTION

The present disclosure relates to UE measurement capability indications for AI/ML dataset construction in 5G NR systems, and in particular methods and apparatus for taking into account UE measurement capabilities when using UE measurement data in AI/ML training datasets.

5G mobile communication technologies define broad frequency bands such that high transmission rates and new services are possible, and can be implemented not only in “Sub 6GHz” bands such as 3.5GHz, but also in “Above 6GHz” bands referred to as mmWave including 28GHz and 39GHz. In addition, it has been considered to implement 6G mobile communication technologies (referred to as Beyond 5G systems) in terahertz bands (for example, 95GHz to 3THz bands) in order to accomplish transmission rates fifty times faster than 5G mobile communication technologies and ultra-low latencies one-tenth of 5G mobile communication technologies.

At the beginning of the development of 5G mobile communication technologies, in order to support services and to satisfy performance requirements in connection with enhanced Mobile BroadBand (eMBB), Ultra Reliable Low Latency Communications (URLLC), and massive Machine-Type Communications (mMTC), there has been ongoing standardization regarding beamforming and massive MIMO for mitigating radio-wave path loss and increasing radio-wave transmission distances in mmWave, supporting numerologies (for example, operating multiple subcarrier spacings) for efficiently utilizing mmWave resources and dynamic operation of slot formats, initial access technologies for supporting multi-beam transmission and broadbands, definition and operation of BWP (BandWidth Part), new channel coding methods such as a LDPC (Low Density Parity Check) code for large amount of data transmission and a polar code for highly reliable transmission of control information, L2 pre-processing, and network slicing for providing a dedicated network specialized to a specific service.

Currently, there are ongoing discussions regarding improvement and performance enhancement of initial 5G mobile communication technologies in view of services to be supported by 5G mobile communication technologies, and there has been physical layer standardization regarding technologies such as V2X (Vehicle-to-everything) for aiding driving determination by autonomous vehicles based on information regarding positions and states of vehicles transmitted by the vehicles and for enhancing user convenience, NR-U (New Radio Unlicensed) aimed at system operations conforming to various regulation-related requirements in unlicensed bands, NR UE Power Saving, Non-Terrestrial Network (NTN) which is UE-satellite direct communication for providing coverage in an area in which communication with terrestrial networks is unavailable, and positioning.

Moreover, there has been ongoing standardization in air interface architecture/protocol regarding technologies such as Industrial Internet of Things (IIoT) for supporting new services through interworking and convergence with other industries, IAB (Integrated Access and Backhaul) for providing a node for network service area expansion by supporting a wireless backhaul link and an access link in an integrated manner, mobility enhancement including conditional handover and DAPS (Dual Active Protocol Stack) handover, and two-step random access for simplifying random access procedure (2-step RACH for NR). There also has been ongoing standardization in system architecture/service regarding a 5G baseline architecture (for example, service based architecture or service based interface) for combining Network Functions Virtualization (NFV) and Software-Defined Networking (SDN) technologies, and Mobile Edge Computing (MEC) for receiving services based on UE positions.

As 5G mobile communication systems are commercialized, connected devices that have been exponentially increasing will be connected to communication networks, and it is accordingly expected that enhanced functions and performances of 5G mobile communication systems and integrated operations of connected devices will be necessary. To this end, new research is scheduled in connection with eXtended Reality (XR) for efficiently supporting AR (Augmented Reality), VR (Virtual Reality), MR (Mixed Reality) and the like, 5G performance improvement and complexity reduction by utilizing Artificial Intelligence (AI) and Machine Learning (ML), AI service support, metaverse service support, and drone communication.

Furthermore, such development of 5G mobile communication systems will serve as a basis for developing not only new waveforms for providing coverage in terahertz bands of 6G mobile communication technologies, multi-antenna transmission technologies such as Full Dimensional MIMO (FD-MIMO), array antennas and large-scale antennas, metamaterial-based lenses and antennas for improving coverage of terahertz band signals, high-dimensional space multiplexing technology using OAM (Orbital Angular Momentum), and RIS (Reconfigurable Intelligent Surface), but also full-duplex technology for increasing frequency efficiency of 6G mobile communication technologies and improving system networks, AI-based communication technology for implementing system optimization by utilizing satellites and AI (Artificial Intelligence) from the design stage and internalizing end-to-end AI support functions, and next-generation distributed computing technology for implementing services at levels of complexity exceeding the limit of UE operation capability by utilizing ultra-high-performance communication and computing resources.

It is an aim of certain examples of the present disclosure to provide approaches for taking into account UE measurement capabilities that may affect training data quality when UE measurement data is used in AI/ML training datasets.

In a first aspect of the present disclosure, provided herein is a method for obtaining measurement data for artificial intelligence or machine learning (AI/ML) model training in a wireless communication system, the method performed by a terminal and comprising: receiving measurement configuration information from a network entity; obtaining the measurement data for AI/ML model training based on the measurement configuration information; and transmitting a measurement report to the network entity, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.

Additionally, or alternatively, the method may further comprise: receiving a measurement capability enquiry from the network entity; and in response to the measurement capability enquiry, transmitting information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.

Additionally, or alternatively, the measurement configuration information may be based on the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.

Additionally, or alternatively, the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training may include information on one or more of terminal type, terminal category, measurement quality, measurement accuracy, measurement validity, measurement periodicity, and a number of attempts to measure a value.

Additionally, or alternatively, the method may further comprise: receiving, from the network entity, information for controlling characteristics of the measurement data for AI/ML model training.

Additionally, or alternatively, the information for controlling characteristics of the measurement data for AI/ML model training may include at least one of information on time period, information on location of the terminal, or information on a measurement method.

In a second aspect of the present disclosure, provided herein is a method for obtaining measurement data for artificial intelligence or machine learning (AI/ML) model training in a wireless communication system, the method performed by a network entity and comprising: transmitting measurement configuration information to a terminal, wherein the measurement configuration information enables the terminal to obtain the measurement data for AI/ML model training; and receiving a measurement report from the terminal, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.

Additionally, or alternatively, the method may further comprise: transmitting a measurement capability enquiry to the terminal; and in response to the measurement capability enquiry, receiving information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.

Additionally, or alternatively, transmitting the measurement configuration information may include: transmitting, to the terminal, the measurement configuration information based on the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.

Additionally, or alternatively, the method may further comprise: transmitting, to the terminal, information for controlling characteristics of the measurement data for AI/ML model training.

Additionally, or alternatively, the method may further comprise: adapting the received measurement data based on the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.

In a third aspect of the present disclosure, provided herein is a terminal for a wireless communication system, the terminal comprising: a transceiver; and a processor coupled to the transceiver and configured to implement a method comprising: receiving measurement configuration information from a network entity; obtaining measurement data for AI/ML model training based on the measurement configuration information; and transmitting a measurement report to the network entity, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.

In a fourth aspect of the present disclosure, provided herein is a network entity for a wireless communication system, the network entity comprising: a transceiver; and a processor coupled to the transceiver and configured to implement a method comprising: transmitting measurement configuration information to a terminal, wherein the measurement configuration information enables the terminal to obtain the measurement data for AI/ML model training; and receiving a measurement report from the terminal, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.

Figure 1 provides a schematic diagram of a UE capability message and its contents in accordance with an example of the present disclosure.

Figure 2 provides a schematic diagram of a UE measurement report message and its contents in accordance with an example of the present disclosure.

Figure 3 provides a schematic diagram of a network entity in accordance with an example of the present disclosure.

Wireless or mobile (cellular) communications networks in which a mobile terminal (UE, such as a mobile handset) communicates via a radio link with a network of base stations, or other wireless access points or nodes, have undergone rapid development through a number of generations. The 3^rd Generation Partnership Project (3GPP) design, specify and standardise technologies for mobile wireless communication networks. Fourth Generation (4G) and Fifth Generation (5G) systems are now widely deployed. In this specification, a User Equipment (UE) may be interchangeably referred to as a terminal, a device, a mobile terminal, a mobile device, a mobile handset, and so on. In this specification, a base station may be interchangeably referred to as a gNB (or gNodeB), an eNB (or eNodeB), a node, an access point, an access node, a Transmission/Reception Point (TRP), a Radio Access Network (RAN), a network device (or network apparatus), and so on.

3GPP standards for 4G systems include an Evolved Packet Core (EPC) and an Enhanced-UTRAN (E-UTRAN: an Enhanced Universal Terrestrial Radio Access Network). The E-UTRAN uses Long Term Evolution (LTE) radio technology. LTE is commonly used to refer to the whole system including both the EPC and the E-UTRAN, and LTE is used in this sense in the remainder of this document. LTE should also be taken to include LTE enhancements such as LTE Advanced and LTE Pro, which offer enhanced data rates compared to LTE.

In 5G systems a new air interface has been developed, which may be referred to as 5G New Radio (5G NR) or simply NR. NR is designed to support the wide variety of services and use case scenarios envisaged for 5G networks, though builds upon established LTE technologies. New frameworks and architectures are also being developed as part of 5G networks in order to increase the range of functionality and use cases available through 5G networks. One such new framework is the use of Artificial Intelligence/Machine Learning (AI/ML) for the optimisation of the operation of 5G networks. However, AI/ML is reliant on training data, and therefore the quality of an AI/ML model is reliant on the quality of the training data on which it is based.

More specifically, AI/ML requires a large amount of data to train the models before applying them in real time (or near real time) as solutions on the ground. The accuracy and the effectiveness of these AI/ML solutions largely depend on the quality of the training data.

In 5G systems training/learning of AI/ML can be performed at the UE and/or at the network. For example, training may be completed or mostly completed at the UE, which is termed as Federated Learning. Alternatively, training may be fully centralized in the network (including the gNBs). Hybrid models of the above two variants also exist. However, regardless of the specific training/learning model used, the quality of the training data can have a significant impact on the performance of the AI/ML model i.e. the quality/accuracy of the inferences output by the AI/ML model.

The content of the following documents is referred to below and/or their content provides useful background information that the following disclosure should be considered in the context of:

- RP-213599, Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR Air Interface R2-2206509, Corrections to stage 2 for NR NTN.

- 3GPP TS 38.331 v17.1.0.

Examples in accordance with the present disclosure will now be described in the context of a 5G wireless communication network comprising at least one or more mobile terminals (or UEs), one or more base stations (or gNB) or Radio Access Network (RAN), and a Core Network (CN). The 5G system may also be considered to be formed from one or more mobile terminals and the network, where the network may comprise one or more network entities (e.g. gNB, Access & Mobility Management Function (AMF), CN etc.). However, it will be understood that the present disclosure is not limited to only 5G system but may be applied to other wireless communication systems in which satellite communications are available. Consequently, references to particular 3GPP constructs in certain examples should not be understood as limiting the ability of examples of the present disclosure to be applied to other wireless communication networks.

The use of Artificial Intelligence/Machine Learning (AI/ML) for the New Radio (NR) air interface has been outlined in RP-213599: Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR Air Interface, in which the following is set out.

Study the 3GPP framework for AI/ML for air-interface corresponding to each target use case regarding aspects such as performance, complexity, and potential specification impact.

Use cases to focus on:

- Initial set of use cases includes:

○ CSI feedback enhancement, e.g., overhead reduction, improved accuracy, prediction [RAN1]

○ Beam management, e.g., beam prediction in time, and/or spatial domain for overhead and latency reduction, beam selection accuracy improvement [RAN1]

○ Positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS conditions [RAN1]

- Finalize representative sub use cases for each use case for characterization and baseline performance evaluations by RAN#98

○ The AI/ML approaches for the selected sub use cases need to be diverse enough to support various requirements on the gNB-UE collaboration levels

[…]

AI/ML model, terminology and description to identify common and specific characteristics for framework investigations:

- Characterize the defining stages of AI/ML related algorithms and associated complexity:

○ Model generation, e.g., model training (including input/output, pre-/post-process, online/offline as applicable), model validation, model testing, as applicable

○ Inference operation, e.g., input/output, pre-/post-process, as applicable

- Identify various levels of collaboration between UE and gNB pertinent to the selected use cases, e.g.,

○ No collaboration: implementation-based only AI/ML algorithms without information exchange [for comparison purposes]

○ Various levels of UE/gNB collaboration targeting at separate or joint ML operation.

- Characterize lifecycle management of AI/ML model: e.g., model training, model deployment , model inference, model monitoring, model updating

- Dataset(s) for training, validation, testing, and inference

- Identify common notation and terminology for AI/ML related functions, procedures and interfaces

- Note: Consider the work done for FS_NR_ENDC_data_collect when appropriate

For the use cases under consideration:

1) Evaluate performance benefits of AI/ML based algorithms for the agreed use cases in the final representative set:

○ Methodology based on statistical models (from TR 38.901 and TR 38.857 [positioning]), for link and system level simulations.

■ Extensions of 3GPP evaluation methodology for better suitability to AI/ML based techniques should be considered as needed.

■ Whether field data are optionally needed to further assess the performance and robustness in real-world environments should be discussed as part of the study.

■ Need for common assumptions in dataset construction for training, validation and test for the selected use cases.

■ Consider adequate model training strategy, collaboration levels and associated implications

■ Consider agreed-upon base AI model(s) for calibration

■ AI model description and training methodology used for evaluation should be reported for information and cross-checking purposes

○ KPIs: Determine the common KPIs and corresponding requirements for the AI/ML operations. Determine the use-case specific KPIs and benchmarks of the selected use-cases.

■ Performance, inference latency and computational complexity of AI/ML based algorithms should be compared to that of a state-of-the-art baseline

■ Overhead, power consumption (including computational), memory storage, and hardware requirements (including for given processing delays) associated with enabling respective AI/ML scheme, as well as generalization capability should be considered.

2) Assess potential specification impact, specifically for the agreed use cases in the final representative set and for a common framework:

○ PHY layer aspects, e.g., (RAN1)

■ Consider aspects related to, e.g., the potential specification of the AI Model lifecycle management, and dataset construction for training, validation and test for the selected use cases

■ Use case and collaboration level specific specification impact, such as new signalling, means for training and validation data assistance, assistance information, measurement, and feedback

○ Protocol aspects, e.g., (RAN2) - RAN2 only starts the work after there is sufficient progress on the use case study in RAN1

■ Consider aspects related to, e.g., capability indication, configuration and control procedures (training/inference), and management of data and AI/ML model, per RAN1 input

■ Collaboration level specific specification impact per use case

○ Interoperability and testability aspects, e.g., (RAN4) - RAN4 only starts the work after there is sufficient progress on use case study in RAN1 and RAN2

■ Requirements and testing frameworks to validate AI/ML based performance enhancements and ensuring that UE and gNB with AI/ML meet or exceed the existing minimum requirements if applicable

■ Consider the need and implications for AI/ML processing capabilities definition

From this study, the present disclose considers issues relevant to at least the following aspects in the context of UE measurements and AI/ML.

- Positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS conditions

- Dataset(s) for training, validation, testing, and inference

- Need for common assumptions in dataset construction for training, validation and test for the selected use cases.

More specifically, currently in mobile and/or wireless communications, UE (or device/terminal/mobile device) measurement reports are used in determining network procedures that impact the given UE only, so the quality of the UE measurement is of little importance outside of the UE. However, when AI/ML is used, a vast amount of UE measurements, collected by different type of UEs, which may have different measurement capabilities and thus produce different quality measurements or measurements with different parameters, will be used to determine overall network policies and behaviours via their use in the training of the AI/ML models. Therefore, these differing measurement capabilities will impact other entities in the network, such as other UEs of network that make use of the AI/ML model trained on the UE measurement data. For example, AI/ML models trained using UE measurement data may be used for handover control and NLOS path detection and thus shortcomings in AI/ML models with have adverse effects on network performance. In the present disclosure, measurement capabilities or indications of measurement quality may include indications of measurement quality, accuracy, validity, and periodicity for example, where the measurements themselves may relate to one or more of signal strength (absolute or relative), signal quality, signal timings, UE location or any other measurement that may be made by a UE and potentially used for AI/ML training. For example, different UEs may have different radio-frequency (RF) reception chain qualities that will influence the accuracy of measurements made on signal strength/quality for example. Therefore, mixing measurement data from UEs with different measurement capabilities may lead to inconsistent AI/ML training datasets and thus potentially reduced quality inferences output by the AI/ML model.

A UE may collect relative or absolute measurements in certain scenarios and the measurement quality can impact AI/ML models using both these measurement types. For example, 'relative' signal strengths or signal quality (Reference Signal Received Power (RSRP) or Reference Signal Received Quality (RSRQ) measurements) from the serving and neighbour gNBs is considered when deciding on handover. Consequently, the actual handover point may differ based on the quality of the measurements provided by the relevant UEs, where different types of UEs may provide different measurement qualities. Therefore, when an AI/ML model is developed to predict the handover point, for example, having the training data set 'biased' towards a certain type of UE due to their measurement capabilities (where the mix of UEs will be different in the actual implementation) will impact the model negatively and thus the output of the model when used to control handover i.e. non-optimal or less optimal handover points. Similarly, 'Absolute' measurements of signal strength can be used, along with the UE location, to determine if a UE is receiving Line of Sight (LOS) or Non LOS (NLOS) signals from the serving and neighbour gNBs. However, when such measurements are used in an AI/ML model, low quality signal strength measurements from some UEs will impact the model negatively, thus leading to poorly NLOS detection. Therefore, in an AI/ML model, high-quality signal strength measurements from some UEs will have to be taken account of and other measurements 'calibrated up' to reflect the actual measurement values in the training data. This will ensure that the AI/ML model training is not 'polluted' by certain low-quality measurements, in situations where the absolute value of the measurements is needed.

In summary, dataset(s) for training AI/ML models preferably need to be constructed based on highly accurate/ high quality measurement data in order to achieve accurate AI/ML operation (e.g. final inference). It is also desirable for the measurements used to form such datasets to have a consistent quality. However, a problem occurs when the measurements used to form the training dataset(s) are of different qualities/have different parameters due to the capabilities or other behaviour of the UEs that collected them.

At present, there is no way for the network entity constructing the training dataset(s) for a given AI/ML model, to know whether the UE or group of UEs that belong to a given training session is/are capable of measuring the desired AI/ML data at a given measurement quality and/or accuracy level/threshold. Likewise, it is not currently possible for the network to determine the quality of measurement reports it receives. Consequently, it is not currently possible for the network to ensure a minimum quality level for training data and/or to account for or compensate for different measurement capabilities/qualities of UEs from which data for forming datasets is obtained. In other words, when an AI/ML relies on UE measurement data and different UEs are expected to provide such data, the impacts of different levels of measurement qualities provided by each UE type needs to be accounted for.

Accordingly, the present disclosure is directed towards the problem of accounting for the impact that UE measurement capabilities have on the quality of training data for AI/ML, and in particular AI/ML models used in the air interface of 5G NR systems or beyond. More specifically, the present disclosure provides several solutions/methods to address the problem of possible training data pollution due to mixing measurements from different UE types (i.e. UEs with different measurement capabilities/parameters).

AI/ML Training Dataset Calibration

To take account of the impact of UE measurement capabilities and the quality/characteristics of the measurements they produce, the present disclosure focusses on the quality of the measurement data a UE (or a wireless device including sensor devices) provides for Artificial Intelligence (AI) or Machine Learning (ML) based solutions, and the use of dataset calibration using information on the data quality. For example, different makes (or categories) of devices will have different capabilities to measure signals (e.g. reference signals) transmitted from the gNBs and hence will report back data with different qualities. Different qualities may also result from different measurement parameters (e.g. accuracy, time period etc.) unrelated to the type of UE. This information on the different qualities can be used to control UEs to perform measurements of a consistent quality and/or calibrate the resulting data (e.g. to a common level) so that adverse effects on the output of the AI/ML model due to inconsistent training data can be reduced. In particular, it is proposed that measurement quality/measurement capabilities are taken into account by the network by configuring UEs to report back their measurement capability or measurement qualities to the network, and the network doing one or more of: configuring measurement rules/parameters to control the characteristics of the measurements performed by the UEs, adjusting received measurements or taking account of their differences before forming a training dataset or when the dataset is being used, and/or adjusting the outputs of the AI/ML model to account for the varying measurement qualities. In other words, an indication of measurement capabilities/quality associated with UEs is provided to the network so that the network can carry out some form of calibration before measurement takes place and/or before or after the data is used to train AI/ML models. In the context of the present application, training data may relate to one or more of a training a model/training session/model updates, such that training data may or may not fully train a model and/or may be partial training data. For example, pre-defined training sessions may be defined by the network in order to train or update AI/ML models. Furthermore, the term network refers to one or more entities on the network side of the 5G systems, such as the gNB, Access & Mobility Management Function (AMF), etc.

More specifically, the proposed approaches are centred around reporting UE measurement capability to the network, before or along with the set of UE measurement data or training outcomes to be used in an AI/ML algorithm. The AI/ML algorithm can be Centralized, Federated, or hybrid, but reporting the UE measurement capability will be useful in all these cases. The AI/ML algorithm/model may be implemented by any suitable network entity and the collection of measurement capabilities, the configuring of measurement rules, the calibration of measurement data, and the aggregation of measurement data may be performed by any suitable network entity (e.g. gNB, AMF, other etc.). In a centralized algorithm where the training is done in the network, this UE measurement capability can be used to control measurement collection and/or calibrate the measurement datasets before using it in the training stage. In Federated or hybrid algorithms, where part or all of the training can be done in the UE/device, reporting the UE measurement capability can help the network to calibrate the training datasets(s) and/or training outcomes. In some cases the network may decide to discard the reported data set(s) considering the UE capability/measurement quality.

In accordance with an example of the present disclosure, the network (including the gNB) may configure AI/ML related data measurements and/or measurements rules (e.g. measurement accuracy-level, reporting periodicity, number of attempts at a certain measurement etc.) at a UE taking account of a UE’s type (i.e. UE’s AI/ML measurement capability). For example, the network may configure the:

- Signal strength measurements of a serving cell and neighbour cells provided by UEs in Handover initiation scenarios.

- Signal strength measurements of a serving cell and neighbour cells provided by UEs at any location of a controlled network (like Industry Internet of Things (IoT)). These measurements can be used for identifying the prevailing LOS/NLOS links at a given location.

In this example, the configuring of one or more UEs that are providing AI/ML measurements is performed prior to the collection of the actual measurements. Therefore, UEs can be configured to provide measurements of a predefined accuracy and at predefined times for example, so that the acquired measurement data is consistent or more consistent across the UEs from which measurements are gathered by the gNB. In other words, the measurement data received by the network has to at least some extent already been calibrated. However, approaches of the present disclosure alternatively or additionally encompass post-processing of measurement data in order to improve consistency between acquired measurement data before its use as training data. For example, such an approach may involve scaling of measurement data taking into account characteristics of the UE or otherwise appropriately weighting data based on UE characteristics.

The network may configure AI/ML measurements, measurement rules, and reporting at the UE and/or adjust received measurements based on the following:

- UE’s subscriber information retrieved, e.g. from Unified Data Management (UDM), that include information on UE’s AI/ML measurement capability, if available.

- Network analytics, e.g. Network Data Analytics Function (NWDAF), assistance information on statistics and/or predictions on UE’s measurement trustability, accuracy (e.g. locations calculation accuracy), and/or other.

- Information on UE AI/ML measurement capability obtained directly from the UE or from any other network (NW) internal or external entity or NW function.

UE measurement capabilities and indications thereof may take any appropriate form, for example, measurement capabilities may be broadly equated to UE type, measurement capabilities may be classified into predefined bands, or specific details (e.g. receiver sensitivity, interference suppression abilities etc., location accuracy) may be provided as measurement capabilities.

Configuration of UE measurements/measurement rules may be based on a desired measurement quality of the network, the measurement capabilities of the relevant UEs or a combination of these. For example, UE measurements may be configured based on the highest achievable quality given the UEs that are performing the measurements. Alternatively, measurements may be configured based on parameters such as power consumption or the quantity/quality of measurements required by the network. In yet another alternative, measurements may be configured to achieve a desired accuracy output from the AI/ML model. In other examples, if UEs with varying capabilities are present, the network may only configure those UEs which can provide higher quality measurements to perform measurements/provide measurement data for AI/ML purposes.

The network may also take a number of other approaches to the configuration of UE measurements/measurement rules. For example, the network may configure one or more measurement rules depending on the training models/sessions that a UE is part of; the network may configure/assign one or more measurement rules for a UE or group of UEs that have the same or similar AI/ML measurement capabilities; the network may configure the UE with different measurement rules depending on the required training data, training session, training model, training model updates, or other factors such as time of day, UE location, etc. (i.e. measurement characteristics required by the network).

Similarly, when post-measurement calibration is used either alone or in combination with pre-measurement configuration, the network may calibrate measurements (e.g. adjust, weight, discard etc.) based on the factors set out above, such as the desired accuracy of the output from the AI/ML models or a required consistency between measurements used to train the AI/ML models.

The appropriate configuration/calibration of UE measurements requires knowledge at the network of UE capabilities. Such knowledge may be obtained by the network (e.g. gNB, AMF, other network entity) when a UE is first registered with the network or may be obtained at any subsequent point in time, such as when measurements are provided to the network or when AI/ML models/parameters are provided to the network (i.e. in the case of federated or hybrid models). For example, the network may request information related to the UE AI/ML measurement capability from the UE, using a UE CAPABILITY TRANSFER procedure. In particular, if the UE supports AI/ML measurements and reporting, the UE may include the information on UE AI/ML measurement capability in a newly defined IE, e.g. UE-CapabilityAI/ML-Meas IE or any other existing IE, where this IE may be included in a UECapabilityInformation message or other suitable message.

Figure 1 provides a message flow diagram illustrating example capability transfer messaging where the UE 102 sends information 108 on a UE AI/ML measurement capability to the network 104 or entity thereof (e.g. gNB, AMF, other etc.) in a UECapabilityInformation message in response to a UECapabilityEnquiry message 106. Such messaging may be performed periodically, when a UE first registers, when AI/ML measurements are requested by the network, or when measurements/AI/ML data is provided by the UE to the network when the UE is RRC_connected.

As noted above, UE capability information is required to be received at the network in order for some form of configuration to take place, whether this be pre or post measurement. In one example, a UE informs the network that it is capable of performing AI/ML measurements and provides information describing these measurements (e.g. measurement quality, accuracy, validity, periodicity, other). If the network has received such a UE capability, the network may configure the UE with suitable measurements (and/or measurements rules) or calibrate received measurements. With respect to the provision of capability information from a UE to the network in the context of AI/ML datasets, the following update may be made to 3GPP TS 38.331 v17.1.0

In some examples, a UE may send to the network information on existing measurements rules (e.g. previously preconfigured). The network may check and update measurement rules if needed.

Alternatively, the UE may send to the network assistance information (i.e. measurement parameters or characteristics of measurements) on performed AI/ML measurements together (or in a separate newly defined or existing signaling /messages). For example, the UE may send ai/ml-MeasParameters-r18 Information Element (IE) in a MeasurementReport message.

Figure 2 provides an illustration of this message exchange where a UE 202 sends to the network 204 or entity thereof (e.g. gNB, AMF, other etc.) a MeasurementReport message 206 including information on the parameters related to the measurements included in the reports, such as quality, validity, periodicity etc.

As noted above, the nature of the UE measurement capability or quality may take many forms, for example, a limited number of categories for the UE measurement quality may be defined, and one of the category numbers (e.g. 1 to 6 or 1 to 10) reported back to the network. The categories may apply to all measurement types of a UE or different categories for different measurement types (e.g. RSRP or RSRQ). Alternatively, specific measurement parameters may be provided to the network, such as receiver sensitivity or interference suppression characteristics. In other examples, merely an indication of a UE category/type may be provided to the network and measurement characteristics of the UE inferred from the category/type.

With respect to measurement reporting from a UE to the network in the context of AI/ML datasets, the following highlighted fields from 3GPP TS 38.331 v17.1.0 may be used.

With respect to measurements that have been obtained by UEs with differing capabilities and thus have different associated qualities (either due to no configuration of the measurement rules or configuration of the measurement rules has only partially reduced differences in measurement quality), a number of approaches may be taken account for these variability between the measurements. For example, measurements from each UE and type of UE may be adjusted/weighted/extrapolated etc. based on their measurement capabilities/parameters under which the measurements were collected.

The network may also take other approaches to handling measurement data/values from UEs, for example, the network may only consider reported measurements from one or more UEs (either pre-determined UEs or determined on measurement characteristics); after receiving UE AI/ML measurement data/reports and/or UE AI/ML measurement capability, the network may indicate (e.g. via further measurement configuration information) to the UE (or group of UEs) to stop measurement (e.g. if the network has collected enough measurement data for the desired training model/session/model updates) and/or stop reporting measurement and/or repeat measurements (e.g. at a different quality, accuracy, or using a different measurement method(s)); the network may discard the reported UE or UEs AI/ML measurements if not provided according to the preconfigured measurement rules or if the network has collected enough measurement data for the desired training model/session/model updates; and the network may only consider AI/ML measurements reported from a given UE or set of UEs and ignore /discard measurements from other UEs (e.g. UEs in a given training session). The network may also implement any combination of these examples.

Alternatively, or additionally, AI/ML models constructed from such variable measurements may be considered to be only fully applicable to UEs with a corresponding measurement capability. For example, in a situation like handover optimization, where 'relative' signal strengths are the key, the problem happens when the mix of differently capable UEs providing the training data is different to the mix of UEs in the implementation of the solution. As an example, considering 2 broad categories of reported measurements from UEs of category A and B - if the training data contains 50% each of A and B type UEs and then in a given situation of actual implementation of the AI/ML solution there is a mix of 80% of A and 20% of B type UEs - there will be a mismatch. However, such drawbacks may be addressed by the network by identifying the % mix of different UE categories in the training data set and in the actual implementation, and then applying appropriate weightings to data from different types of UEs in the training data set. For this to happen, the UE measurement capability or type may be reported back to the network, and the UEs categorized (possibly to bands) based on their measurement capability. However, approaches to the post-collection adjustment of measurements may be configured to have a higher complexity or lower complexity dependent on the desired level of consistency between the resulting adjusted measurements.

In some situations, like the NLOS path identification, the absolute values of signal strengths matter. If two types of UEs (A and B) report different signal strengths and A value is much closer to the actual measurement (identified through the measurement quality of the UE type), the network can calibrate (up) the measurements received through type B UEs and then use this calibration both in training and in actual implementation. For this solution also, the measurement category/characteristics has to be reported back to the network and the network has to make an assessment on the level of calibration to apply, depending on the best quality UE measurements it will receive.

Handover Optimisation

The traditional handover procedure can be termed as reactive, the serving gNB responds to decreasing measurement reports from a UE and when this falls below a threshold, the handover procedures are initiated. The (Secondary) Synchronisation Signal Reference Signal Received Quality (SS-RSRQ) or Channel State Information (CSI)-RSRQ measurements in 5G can be used to initiate the handover procedures. In a single UE (non AI/ML) scenario, the handover for the UE needs to be determined as per the measurements reported by the said UE, so quality of the signal measurements does not significantly impact the decision. However, in an AI/ML based predictive scenario, much finer details like sudden signal blockages or spikes of interference can be identified at a particular cell border and the handovers can be optimized considering these effects. In order to train these algorithms, RSRQ data from UEs/devices crossing a particular cell boarder can be accumulated, and thus the quality of such measurements is important.

The measurements of signal strengths and particularly the interference levels will depend on the quality of the RF chains in a particular device. In particular, interference can occur as co-channel and cross-channel and the quality of the out-of-band Radio Frequency (RF) filters in the UE will determine how much cross-channel interference is captured by the UE/device. Hence some calibration of this measurements, when they are used as input data to train AI/ML algorithms, will be needed and for this reporting the quality of the UE measurements (as part of the UE capability) will be useful. Such calibration can be based on information on parameters of the UE (i.e. measurement capabilities) that have been provided to the network from the UE as described above. Alternatively, via the setting of measurement parameters/rules by the network based on knowledge of the UE capabilities, consistent interference level measurements may be obtained, reducing the need for further calibration of measurements received from multiple UEs.

NLOS Path Detection

Dense and ultra-dense gNB deployments will be common for 5G Advance (5G-adv) networks, in particular to support industry IoT sites and urban hotspots. Precise localization will be a requirement in some of these networks, for example in Indoor factory (Industry IoT) sites to locate moving objects and workers. One of the key requirements for precise localization is to have a sufficient number of Line of Sight (LOS) links to a particular UE under localization/tracking. In a dynamic environment, the LOS/Non LOS (NLOS) links at a given location will vary. Hence it will be very useful to have an AI/ML based solution to identify NLOS links at a given location, so they can be excluded from the usage in localization/tracking algorithms.

Such AI/ML solutions can use the Reference Signal Received Power (RSRP) measured and reported by a UE for the different gNBs (serving and neighbour) from a given location as training data. Again, the quality AI/ML model will be dependent on the quality of the reported measurements and the quality of the reported measurements will depend on the quality of the RF chains in a particular device. When data from multiple UEs/devices or sensors are used in such AI/ML algorithms, reporting the quality of the UE measurements (as part of the UE capability) will be useful and measurements configured/calibrated as set out above in order to improve the quality of training data for the AI/ML model.

Although the approaches set out above have specified particular types of measurements for specific purposes, the approaches of this disclosure are not limited to these. For example, the approaches may be applied to any form of data that is provided by a UE for AI/ML training purposes and which may be affected by characteristics of the UE, so that variability in a wide range of UE data can be compensated for and resulting datasets appropriately calibrated. Furthermore, the approaches set out above may be used alone or in combination with each other, for example, any combination of UE capability reporting, measurement rule adjustment, and measurement calibration may be used.

Further examples in accordance with the present disclosure are set out in the following numbered clauses, where these examples may be combined with one or more of the approaches set out above unless stated otherwise.

1. A method of a (mobile) terminal for obtaining measurement data for AI/ML model training in a 5G NR or beyond communications system comprising a base station and one or more (mobile) terminals, the method comprising:

transmitting, from the (mobile) terminal to the base station, information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training;

measuring, at the (mobile) terminal, one or more values associated with the data for AI/ML model training; and

transmitting, from the (mobile) terminal to the base station, the measured values.

2. The method of clause 1, wherein the method further comprises receiving, at the (mobile) terminal from the base station, a measurement capability enquiry, and transmitting the information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training in response to the measurement capability enquiry.

3. The method of clauses 1 or 2, wherein the method further comprises receiving measurement configuration information from the base station, and the measuring of the one or more values associated with the data for AI/ML model training is based on the measurement configuration information.

4. The method of clause 3, wherein the measurement configuration information is based on the information on one or more measurement capabilities of the (mobile) terminal.

5. The method of clauses 3 or 4, wherein the measurement configuration information includes information on one or more of measurement quality, measurement accuracy, measurement validity, and measurement periodicity.

6. The method of clause 1, wherein the information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training is transmitted to the base station along with the measured values.

7. The method of any preceding clause, wherein the information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training includes information on one or more of (mobile) terminal type, (mobile) terminal category, measurement quality, measurement accuracy, measurement validity, measurement periodicity, and a number of attempts to measure a value (e.g. a reference signal).

8. The method of any preceding clause, wherein the one or more values include one or more of a location of the (mobile) terminal, a received signal strength, a received signal quality, and signal timing information.

9. A method of a base station for obtaining measurement data for AI/ML model training in a 5G NR or beyond communications system comprising the base station and one or more (mobile) terminals, the method comprising:

receiving, from the (mobile) terminal at the base station, information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training; and

receiving, from the (mobile) terminal at the base station, one or more values associated with the data for AI/ML model training measured by the (mobile) terminal.

10. The method of clause 9, wherein the method further comprises transmitting from the base station to the (mobile) terminal, measurement configuration information based on the information on one or more measurement capabilities of the (mobile) terminal.

11. The method of clause 10, wherein the measurement configuration information includes information on one or more of measurement quality, measurement accuracy, measurement validity, and measurement periodicity

12. The method of clauses 9 or 10, wherein the measurement configuration information is based on one or more of an AI/ML model or training session associated with the (mobile) terminal, a measurement characteristic required by the base station or other network entity, and measurement capabilities of at least one other (mobile) terminal served by the base station.

13. The method of clause 9, wherein the information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training is received by the base station or other network entity along with the measured values.

14. The method of any of clauses 9 to 13, wherein the information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training includes information on one or more of (mobile) terminal type, (mobile) terminal category, measurement quality, measurement accuracy, measurement validity, and measurement periodicity.

15. The method of clause 14, wherein the base station or other network entity adapts the received measured values based on the information on one or more measurement capabilities of the (mobile) terminal with respect to data for AI/ML model training

16. The method of any of clauses 9 to 15, wherein the one or more values include one or more of a location of the (mobile) terminal, a received signal strength, a received signal quality, and signal timing information.

17. The method of any of clauses 1 to 16, wherein the base station or other network entity trains an AI/ML model based on the received one or more values.

18. The method of any of clauses 9 to 17, wherein the base station or other network entity trains an AI/ML model based on the received one or more values if the (mobile) terminal is included in a predefined group of (mobile) terminals.

19. The method of clause 10, wherein the base station or other network entity discards one or more of the received values if the one or more of the received values are not in accordance with the measurement configuration information.

20. The method of any of clauses 9 to 19, wherein the method further comprises, in response to receiving the one or more values, transmitting from the base station to the (mobile) terminal, an indication of one or more of:

the (mobile) terminal should stop further measurements of values associated with the data for AI/ML model training,

the (mobile) terminal should stop reporting measured values associated with the data for AI/ML model training,

the (mobile) terminal should repeat the measurement of one or more values associated with the data for AI/ML model training, and

the (mobile) terminal should perform further measurement of one or more values associated with the data for AI/ML model training based on updated measurement configuration information.

21. A (mobile) terminal for a 5G or beyond communications systems, wherein the (mobile) terminal is configured to implement the method of any of clauses 1 to 8.

22. A base station for a 5G or beyond communications system, wherein the base station is configured to implement the method of any of clauses 9 to 20.

23. A 5G or beyond communications system comprising one or more (mobile) terminals, a base station, and other core network elements, wherein the 5G communications system is configured to implement the method of any of clauses 1 to 22.

A UE which is arranged to operate in accordance with any of the examples of the present disclosure described above includes a transmitter arranged to transmit signals to one or more RANs, including but not limited to a satellite network and a 3GPP RAN such as 5G NR network; a receiver arranged to receive signals from one or more RANs and other UEs; and a controller arranged to control the transmitter and receiver and to perform processing in accordance with the above described methods. The transmitter, receiver, and controller may be separate elements, but any single element or plurality of elements which provide equivalent functionality may be used to implement the examples of the present disclosure described above.

Figure 3 is a block diagram of an exemplary network entity that may be used in the implementation of the examples of the present disclosure. For example, the UE, entities of the network-side, core network or RAN (e.g. eNB, gNB or satellite) may be provided in the form of the network entity illustrated in Figure 3. The skilled person will appreciate that a network entity may be implemented, for example, as a network element on a dedicated hardware, as a software instance running on a dedicated hardware, and/or as a virtualised function instantiated on an appropriate platform, e.g. on a cloud infrastructure.

The entity 300 comprises a processor (or controller) 301, a transmitter 303 and a receiver 305. The receiver 305 is configured for receiving one or more messages from one or more other network entities, for example as described above. The transmitter 303 is configured for transmitting one or more messages to one or more other network entities, for example as described above. The processor 301 is configured for performing one or more operations, for example according to the operations as described above.

The techniques described herein may be implemented using any suitably configured apparatus and/or system. Such an apparatus and/or system may be configured to perform a method according to any aspect, embodiment, example or claim disclosed herein. Such an apparatus may comprise one or more elements, for example one or more of receivers, transmitters, transceivers, processors, controllers, modules, units, and the like, each element configured to perform one or more corresponding processes, operations and/or method steps for implementing the techniques described herein. For example, an operation/function of X may be performed by a module configured to perform X (or an X-module). The one or more elements may be implemented in the form of hardware, software, or any combination of hardware and software.

A particular network entity may be implemented as a network element on a dedicated hardware, as a software instance running on a dedicated hardware, and/or as a virtualised function instantiated on an appropriate platform, e.g. on a cloud infrastructure.

It will be appreciated that examples of the present disclosure may be implemented in the form of hardware, software or any combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage, for example a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape or the like.

It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs comprising instructions that, when executed, implement certain examples of the present disclosure. Accordingly, certain examples provide a program comprising code for implementing a method, apparatus or system according to any example, embodiment, aspect and/or claim disclosed herein, and/or a machine-readable storage storing such a program. Still further, such programs may be conveyed electronically via any medium, for example a communication signal carried over a wired or wireless connection.

Throughout the description and claims of this specification, the words "comprise" and "contain" and variations of them mean "including but not limited to", and they are not intended to (and do not) exclude other components, integers or steps. Throughout the description and claims of this specification, the singular encompasses the plural unless the context otherwise requires. In particular, where the indefinite article is used, the specification is to be understood as contemplating plurality as well as singularity, unless the context requires otherwise.

Features, integers or characteristics described in conjunction with a particular aspect, embodiment or example of the present disclosure are to be understood to be applicable to any other aspect, embodiment or example described herein unless incompatible therewith. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive. The disclosure is not restricted to the details of any foregoing embodiments. Examples of the present disclosure extend to any novel one, or any novel combination, of the features disclosed in this specification (including any accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.

The reader's attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

The above embodiments are to be understood as illustrative examples of the present disclosure. Further embodiments are envisaged. It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be used without departing from the scope of the invention, which is defined in any accompanying claims.

Claims

A method for obtaining measurement data for artificial intelligence or machine learning (AI/ML) model training in a wireless communication system, the method performed by a terminal and comprising:

receiving measurement configuration information from a network entity;

obtaining the measurement data for AI/ML model training based on the measurement configuration information; and

transmitting a measurement report to the network entity, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.
The method of claim 1, wherein the method further comprises:

receiving a measurement capability enquiry from the network entity; and

in response to the measurement capability enquiry, transmitting information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.
The method of claim 2, wherein the measurement configuration information is based on the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.
The method of claim 2, wherein the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training includes information on one or more of terminal type, terminal category, measurement quality, measurement accuracy, measurement validity, measurement periodicity, and a number of attempts to measure a value.
The method of claim 1, wherein the method further comprises:

receiving, from the network entity, information for controlling characteristics of the measurement data for AI/ML model training.
The method of claim 5, wherein the information for controlling characteristics of the measurement data for AI/ML model training includes at least one of information on time period, information on location of the terminal, or information on a measurement method.
A method for obtaining measurement data for artificial intelligence or machine learning (AI/ML) model training in a wireless communication system, the method performed by a network entity and comprising:

transmitting measurement configuration information to a terminal, wherein the measurement configuration information enables the terminal to obtain the measurement data for AI/ML model training; and

receiving a measurement report from the terminal, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.
The method of claim 7, wherein the method further comprises:

transmitting a measurement capability enquiry to the terminal; and

in response to the measurement capability enquiry, receiving information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.
The method of claim 8, wherein transmitting the measurement configuration information includes:

transmitting, to the terminal, the measurement configuration information based on the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.
The method of claim 8, wherein the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training includes information on one or more of terminal type, terminal category, measurement quality, measurement accuracy, measurement validity, measurement periodicity, and a number of attempts to measure a value.
The method of claim 7, wherein the method further comprises:

transmitting, to the terminal, information for controlling characteristics of the measurement data for AI/ML model training.
The method of claim 11, wherein the information for controlling characteristics of the measurement data for AI/ML model training includes at least one of information on time period, information on location of the terminal, or information on a measurement method.
The method of claim 8, wherein the method further comprises:

adapting the received measurement data based on the information on one or more measurement capabilities of the terminal with respect to data for AI/ML model training.
A terminal for a wireless communication system, the terminal comprising:

a transceiver; and

a processor coupled to the transceiver and configured to implement a method comprising:

receiving measurement configuration information from a network entity;

obtaining measurement data for AI/ML model training based on the measurement configuration information; and

transmitting a measurement report to the network entity, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.
A network entity for a wireless communication system, the network entity comprising:

a transceiver; and

a processor coupled to the transceiver and configured to implement a method comprising:

transmitting measurement configuration information to a terminal, wherein the measurement configuration information enables the terminal to obtain the measurement data for AI/ML model training; and

receiving a measurement report from the terminal, the measurement report including the measurement data for AI/ML model training and information on a quality of the measurement data for AI/ML model training.