EP4690002A1

EP4690002A1 - Ai/ml model selection based on model drift detection

Info

Publication number: EP4690002A1
Application number: EP24719320.4A
Authority: EP
Inventors: Zhan Zhang; Yufei Blankenship
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2023-04-07
Filing date: 2024-04-05
Publication date: 2026-02-11
Also published as: WO2024210814A1

Abstract

A method, system and apparatus are disclosed. An example user equipment (UE) is configured to communicate with a network node. The UE is configured to determine a monitoring metric for each one of a plurality of machine learning, ML, models. The UE is configured to determine, based on the monitoring metrics, whether to implement one of the plurality of ML models. The UE is configured to communicate with the network node based on the determination of whether to implement one of the plurality of ML models.

Description

AI/ML MODEL SELECTION BASED ON MODEL DRIFT DETECTION

TECHNICAL FIELD

The present disclosure relates to wireless communications, and in particular, to autoencoder-based artificial intelligence (AI)/machine learning (ML) model selection based on drift detection.

BACKGROUND

The Third Generation Partnership Project (3GPP) has developed and is developing standards for Fourth Generation (4G) (also referred to as Long Term Evolution (LTE)) and Fifth Generation (5G) (also referred to as New Radio (NR)) wireless communication systems. Such systems provide, among other features, broadband communication between network nodes, such as base stations, and user equipment (UE), as well as communication between network nodes and between UEs. The 3GPP is also developing standards for Sixth Generation (6G) wireless communication networks.

AI/ML modeling

Machine learning capabilities of Al may provide advantageous policies or strategies directly based on data instead of human logic, symbolic modeling, and analysis. AI/ML enabled solutions may, for instance, employ data-driven learning approaches where the models learn the underlying data distribution and relationships between the inputs and outputs, e.g., without the need for understanding the underlying complex processes. ML has been found to be an effective tool in radio signal-based positioning. For instance, 3GPP has been investigating AI/ML-based positioning methods for achieving high positioning accuracy, e.g., in challenging deployment scenarios.

Positioning based on radio signal measurements data learning

3GPP study items on artificial intelligence / machine learning (AI/ML) based methods for indoor position have been in progress. FIG. l is a diagram which illustrates an example scenario of radio propagation. As illustrated in FIG. 1, different radio propagations may result in different channel features, such as channel coherent bandwidth, channel variation over time and space, etc. One notable feature is that the channel may become rich with multipath characteristics in indoor settings, especially when the indoor setting is densely scattered with so-called clutters, such as machines, assembly lines, and storage shelves, etc., such as in an indoor factory. Line of sight (LOS) radio propagation path between the network node (a radio base station, Transmission-Reception Point (TRP), etc.) and the UE (e.g., user terminal or UE) is seldomly available. The actual radio propagation environment may shape the channel states to be spatially selective and dominated by non-line-of-sight (NLOS) radio paths. Hence, a conventional triangularization or trilateration method assuming all radio paths being LOS may have an unsatisfactory performance.

Thus, AI/ML based positioning methods are under development to improve positioning accuracy, especially for the heavy NLOS environments where the conventional methods fail. Due to constant time-varying environment change, positioning estimation models including machine learning (ML) model typically face a possibility of performance drift.

Model monitoring

Positioning integrity is a measure of trust in the accuracy of the position-related estimation/data provided by the positioning system and the ability to provide timely and valid warnings to the Location Services (LCS) client when the positioning system does not fulfil the condition for intended operation. Integrity focuses on the tail of the positioning error distribution (i.e., the rare events), and aims to keep the probability of hazardous events extremely low. For example, <10'⁷/hr Target Integrity Risk (TIR) translates to one failure permitted every 10 million hours (equivalent to 1142 years approximately).

Hence, to secure positioning integrity, it may be necessary to monitor the pretrained ML model for positioning for possible drifts in performance, due, e.g., to radio environment changes, device failure, interference, etc., which may cause measurement distortions on radio signals.

Radio propagation environments may be constantly changing. For example, in the indoor factory scenario, objects may move around, enter, or exit an area, or indoor furniture may change location, etc. This may cause changes to the measurement data set and the relationships between AI/ML model inputs and their corresponding model outputs. This may further cause drift of Al model. True values of the model output may also be referred to as ground truth labels, or simply “labels” for short, which is a concept often used in training data collection for supervised learning.

Models obtained using the auto-encoder structures

Auto-encoder (AE) is a general terminology for a kind of neural network architecture, where the output of the of the network is (ideally) a regenerated copy of the input data. AEs are useful architectures for unsupervised learning and semi-supervised learning applications. AEs typically exhibit two main components: (i) the encoder g(.), (ii) the decoder f(.). The encoder maps the input data X to the hidden state h (also referred to as coded info), while the decoder strives to map h back to X. When training the AE, the goal is to achieve (as much as is possible), the following: decoder output X' = encoder input X. In other words:

• The encoder function can be described as: g(X) = h,

• The decoder function can be described as: f(K) = X

The auto-encoder structure may be applied to the use case of UE positioning, when there are no or very few labelled training data. In the following, the fingerprinting-based positioning method with the auto-encoder structure is described.

A fingerprinting-based positioning method refers to exploiting a computing graph where the model inference is the UE’s coordinates represented as [x, y, z].

Firstly, as illustrated in the process diagram of FIG. 2, which illustrates an example fingerprint-based positioning machine learner, the learner-A may build a computing graph to infer a position coordinate [x, y, z], which corresponds to a RSRP vector measurement by multiple TRPs (network nodes).

Secondly, a reverse ML function may be built that maps a position [x, y, z] to a RSRP vector. As illustrated in the process diagram of FIG. 3, which illustrates an example Fingerprint-based CSI (RSRP vector) machine learner, with a same data set of CSI (a RSRP vector) and its corresponding position coordinate, a second model may be learned to infer CSI (RSRP vector) corresponding to a position coordinate [x, y, z].

The first estimator (inferring position coordinates) and second estimator (inferring CSI (RSRP vector) may be connected sequentially as an autoencoder, in which the encoder is a position estimator while the decoder is a CIR (RSRP vector) estimator, owing to their converse relationship of input and output variables. The dimension of position coordinate vector (e.g., 3 dimensions) may be much smaller than the RSRP vector (e.g., 18 RSRP values as measured by 18 TRPs/network nodes), hence, this autoencoder structure becomes an undercomplete autoencoder, which helps to conduct a semi-supervised learning, as depicted in the process diagram of FIG. 4, which illustrates an example autoencoder learning scheme.

Recent investigation of AI/ML-based UE positioning has indicated that the positioning accuracy may be improved substantially compared to conventional positioning methods, especially in deployment scenarios dominated by NLOS. However, for some UE-hosted AI/ML fingerprinting methods, if the Al model is trained with a dataset collected at a specific radio propagation environment, it may typically encounter a severe generalization problem, i.e., the model performance deteriorates when the UE experiences an environment different from that of training. The environment change can be caused by movements of objects in the surroundings, and/or the movement of UE itself (e.g., the UE moves from outdoor to indoor, or moves from one room to another room).

When the model performance deteriorates, a model update is one way to maintain the required model performance. However, online re-training or (fine-)tuning in real time is often not feasible. This may severely impact the positioning accuracy integrity of AI/ML based positioning methods. Therefore, existing systems may lack configurations for mitigating such generalization issues and ensuring continuous service of high accuracy positioning of the target UE.

SUMMARY

Some embodiments advantageously provide methods, systems, and apparatuses for supporting configurations for autoencoder-based AI/ML model selection based on drift detection.

In some systems, a significant cause of the generalization issue of an AI/ML model is that the model becomes inappropriate or inaccurate when the mapping relationship between model inputs and model outputs has changed, as compared to the mapping learned from the training dataset. In order words, model drift may have occurred, which may make the previously trained model unsuitable for the new environment (e.g., changing channel conditions, etc.). If the current deployment scenario differs substantially from the training scenario of the activated AI/ML model, the model may need to be updated to a new model.

One method of updating the model is to use model switching. For a UE-side model, if a UE is expected to experience multiple different deployment scenarios, then multiple AI/ML models can be pre-trained and stored in and/or provided to the UE, where an AI/ML model is trained for a targeted deployment scenario among many scenarios and models. In this case, multiple AI/ML models may be available to the UE to select on demand.

In some embodiments of the present disclosure, whenever training an inference unit (i.e., Encoder) to obtain position coordinates (e.g., [x, y, z]) from the model input of radio signal measurement (e.g., RSRP vector), a reverse inference unit (i.e., Decoder) may be trained as well, using the same training dataset (e.g., where the input-output relationship is reversed). The Decoder may map position coordinates (e.g., [x, y, z]) back to an estimation of the radio signal measurement (e.g., RSRP vector).

In some embodiments, for each inference unit (i.e., Encoder), a corresponding Decoder may be trained and stored and/or received (e.g., by the UE and/or network node). These two inference units (i.e., Encoder, Decoder) may be concatenated to form an autoencoder inference unit. When the Encoder is deployed in the field, the corresponding Decoder may be used for model monitoring and model selection. Using this method and/or configuration, model monitoring and selection may be accomplished without requiring labelled data.

The multiple auto-encoder models may be accessible by the deployed UE, for example, pre-stored on the UE, and/or sent from one UE to another UEWD and/or or provided by a network node to the UE; or retrieved over-the-top via downloading from a model storage entity (e.g., a cloud-based entity, node, server, and/or host computer). In some embodiments, a model storage entity may be located either on the network side, on the UE side, or at a neutral side.

In some embodiments, a UE during operation may be configured to choose one or multiple of the Al autoencoder models, and may be configured to conduct model drift detection using a model monitoring method which may produce one or more model monitoring metrics. Based on the model monitoring metric(s), the UE may be configured to select the best position inference unit (i.e., an Encoder) for the current scenario, e.g., the position inference unit that has the best model monitoring metric(s).

While in operation, a UE may be configured to perform model monitoring continuously, or periodically, and/or when triggered. When a model drift alarm is sounded, the UE may prepare to switch to another model, where the preparation stage may include the model selection step.

In embodiments of the present disclosure, a practical model selection may be supported that utilizes methods similar to model drift detection. This may substantially mitigate some generalization issues in some systems.

Embodiments of the present disclosure may be described with respect to a UE side model as an example scenario, but it is to be understood that the same or similar configurations, methodologies, and procedures described herein may be applied to models deployed elsewhere, e.g., a network side model deployed on a network node, on a cloudbased node, etc. Embodiments of the present disclosure may provide one or more model monitoring methods for enabling a UE, network node, etc. to make model management decisions, including model selection, model updating, model activation, and/or model deactivation, as follows:

• Before a candidate model is available be activated, model monitoring methods may be applied (e.g., by a UE, by a network node, etc.) to verify and confirm that the candidate model is expected to work properly in the current scenario and/or configuration.

• When there are multiple candidate models available for a given functionality, model monitoring methods may be applied (e.g., by a UE, by a network node, etc.) to each (i.e., to one or more of) of the candidate models. The candidate model(s) that has/have the best model monitoring metric(s), and/or which have metrics which exceed one or more preconfigured thresholds, may be recommended/identified for model selection.

• If none of the currently available candidate models have sufficiently good model monitoring metrics (e.g., fail to meet a minimum metric threshold), then a decision may be made (e.g., by a UE, by a network node, etc.) to deactivate the AEML model for the given functionality, and a fall-back (e.g., non-AI/ML method) may be used (e.g., by a UE, by a network node, etc.) instead.

In the above example embodiments, a variety of model monitoring methods may be used. In some embodiments, the model monitoring method may not require ground truth labeling for data seen during model deployment. As a non-limiting example, model monitoring may be performed (e.g., by a UE, by a network node, etc.) using a Decoder associated with the AI/ML model being monitored, which may enable model monitoring and therefore the model selection decision-making, where labeled data is either costly or impractical to obtain, and/or such labeling may suffer from low accuracy. Embodiments of the present disclosure thus may support one or more model management procedures.

As described herein, one or more of the model management functionalities may be supported by AI/ML techniques and for a variety of applications. One non-limiting example is AI/ML based UE positioning, which is used in the descriptions below to illustrate the methodologies and procedures, but embodiments of the present disclosure may be applied to a variety of other AI/ML applications.

According to one aspect of the present disclosure, a user equipment, UE is provided. UE is configured to communicate with a network node. UE is configured to determine a monitoring metric for each one of a plurality of machine learning, ML, models. UE is configured to determine, based on the monitoring metrics, whether to implement one of the plurality of ML models. UE is configured to communicate with the network node based on the determination of whether to implement one of the plurality of ML models.

According to one or more embodiments of this aspect, determining whether to implement one of the plurality of ML models includes determining not to implement any of the plurality of ML models.

According to one or more embodiments of this aspect, determining whether to implement one of the plurality of ML models includes selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

According to one or more embodiments of this aspect, determining whether to implement one of the plurality of ML models includes comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of: determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

According to one or more embodiments of this aspect, the UE is further configured to: detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

According to another aspect of the present disclosure, a method performed by a UE configured to communicate with a network node is provided. The method includes: determining a monitoring metric for each one of a plurality of machine learning, ML, models; determining, based on the monitoring metrics, whether to implement one of the plurality of ML models; and communicating with the network node based on the determination of whether to implement one of the plurality of ML models.

According to one or more embodiments of this aspect, the method further includes: detecting that the implemented one of the plurality of ML models has drifted; and determining whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

According to another aspect of the present disclosure, a network node is provided. Network node is configured to communicate with a UE. Network node is configured to determine a monitoring metric for each one of a plurality of machine learning, ML, models. Network node is configured to determine, based on the monitoring metrics, whether to implement one of the plurality of ML models. Network node is configured to communicate with the UE based on the determination of whether to implement one of the plurality of ML models.

According to one or more embodiments of this aspect, the network node is further configured to: detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

According to another aspect of the present disclosure, a method performed by a network node configured to communicate with a UE is provided. The method includes: determining a monitoring metric for each one of a plurality of machine learning, ML, models; determining, based on the monitoring metrics, whether to implement one of the plurality of ML models; and communicating with the UE based on the determination of whether to implement one of the plurality of ML models.

According to one or more embodiments of this aspect, the method further includes: detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present embodiments, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:

FIG. 1 is a schematic diagram illustrating an example scenario of radio propagation;

FIG. 2 is a flowchart of an example autoencoder process; FIG. 3 is a flowchart of another example autoencoder process;

FIG. 4 is a flowchart of another example autoencoder process;

FIG. 5 is a schematic diagram of an example network architecture illustrating a communication system connected via an intermediate network to a host computer according to the principles in the present disclosure;

FIG. 6 is a block diagram of a network node in communication with a user equipment over a wireless connection according to some embodiments of the present disclosure;

FIG. 7 is a flowchart of an example process in a UE for supporting autoencoderbased AI/ML model selection based on drift detection according to some embodiments of the present disclosure;

FIG. 8 is a flowchart of another example process in a UE for supporting autoencoder-based AI/ML model selection based on drift detection according to some embodiments of the present disclosure;

FIG. 9 is a flowchart of an example process in a network node for supporting autoencoder-based AI/ML model selection based on drift detection according to some embodiments of the present disclosure;

FIG. 10 is a flowchart of another example process in a network node for supporting autoencoder-based AI/ML model selection based on drift detection according to some embodiments of the present disclosure;

FIG. 11 is a flowchart of yet another example process in a UE according to some embodiments of the present disclosure;

FIG. 12 is a flowchart of yet another example process in a network node according to some embodiments of the present disclosure; and

FIG. 13 is a flowchart and state transition diagram a model selection procedure according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

Before describing in detail example embodiments, it is noted that the embodiments reside primarily in combinations of apparatus components and processing steps related to supporting autoencoder-based AI/ML model selection based on drift detection. Accordingly, components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. Like numbers refer to like elements throughout the description.

As used herein, relational terms, such as “first” and “second,” “top” and “bottom,” and the like, may be used solely to distinguish one entity or element from another entity or element without necessarily requiring or implying any physical or logical relationship or order between such entities or elements. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

In embodiments described herein, the joining term, “in communication with” and the like, may be used to indicate electrical or data communication, which may be accomplished by physical contact, induction, electromagnetic radiation, radio signaling, infrared signaling or optical signaling, for example. One having ordinary skill in the art will appreciate that multiple components may interoperate and modifications and variations are possible of achieving the electrical and data communication.

In some embodiments described herein, the term “coupled,” “connected,” and the like, may be used herein to indicate a connection, although not necessarily directly, and may include wired and/or wireless connections.

The term “network node” used herein can be any kind of network node comprised in a radio network which may further comprise any of base station (BS), radio base station, base transceiver station (BTS), base station controller (BSC), radio network controller (RNC), g Node B (gNB), evolved Node B (eNB or eNodeB), Node B, multistandard radio (MSR) radio node such as MSR BS, multi -cell/multicast coordination entity (MCE), integrated access and backhaul (IAB) node, relay node, donor node controlling relay, radio access point (AP), transmission points, transmission nodes, Remote Radio Unit (RRU) Remote Radio Head (RRH), a core network node (e.g., mobile management entity (MME), self-organizing network (SON) node, a coordinating node, positioning node, MDT node, etc.), an external node (e.g., 3rd party node, a node external to the current network), nodes in distributed antenna system (DAS), a spectrum access system (SAS) node, an element management system (EMS), etc. The network node may also comprise test equipment. The term “radio node” used herein may be used to also denote a user equipment (UE) such as a wireless device (WD) or a radio network node.

In some embodiments, the non-limiting terms wireless device (WD) or a user equipment (UE) are used interchangeably. The UE herein can be any type of wireless device capable of communicating with a network node or another UE over radio signals, such as wireless device (WD). The UE may also be a radio communication device, target device, device to device (D2D) WD, machine type WD or WD capable of machine to machine communication (M2M), low-cost and/or low-complexity WD, a sensor equipped with WD, Tablet, mobile terminals, smart phone, laptop embedded equipped (LEE), laptop mounted equipment (LME), USB dongles, Customer Premises Equipment (CPE), an Internet of Things (loT) device, or a Narrowband loT (NB-IOT) device, etc.

Also, in some embodiments the generic term “radio network node” is used. It can be any kind of a radio network node which may comprise any of base station, radio base station, base transceiver station, base station controller, network controller, RNC, evolved Node B (eNB), Node B, gNB, Multi-cell/multicast Coordination Entity (MCE), IAB node, relay node, access point, radio access point, Remote Radio Unit (RRU) Remote Radio Head (RRH).

Note that although terminology from one particular wireless system, such as, for example, 3GPP LTE and/or New Radio (NR), may be used in this disclosure, this should not be seen as limiting the scope of the disclosure to only the aforementioned system. Other wireless systems, including without limitation Wide Band Code Division Multiple Access (WCDMA), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband (UMB) and Global System for Mobile Communications (GSM), may also benefit from exploiting the ideas covered within this disclosure.

Note further, that functions described herein as being performed by a UE or a network node may be distributed over a plurality of UEs and/or network nodes. In other words, it is contemplated that the functions of the network node and UE described herein are not limited to performance by a single physical device and, in fact, can be distributed among several physical devices.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

Some embodiments provide configurations for supporting autoencoder-based AI/ML model selection based on drift detection. Referring again to the drawing figures, in which like elements are referred to by like reference numerals, there is shown in FIG. 5 a schematic diagram of a communication system 10, according to an embodiment, such as a 3GPP-type cellular network that may support standards such as LTE and/or NR (5G), which comprises an access network 12, such as a radio access network, and a core network 14. The access network 12 comprises a plurality of network nodes 16a, 16b, 16c (referred to collectively as network nodes 16), such as NBs, eNBs, gNBs or other types of wireless access points, each defining a corresponding coverage area 18a, 18b, 18c (referred to collectively as coverage areas 18). Each network node 16a, 16b, 16c is connectable to the core network 14 over a wired or wireless connection 20. A first user equipment (UE) 22a located in coverage area 18a is configured to wirelessly connect to, or be paged by, the corresponding network node 16a. A second UE 22b in coverage area 18b is wirelessly connectable to the corresponding network node 16b. While a plurality of UEs 22a, 22b (collectively referred to as UEs 22) are illustrated in this example, the disclosed embodiments are equally applicable to a situation where a sole UE is in the coverage area or where a sole UE is connecting to the corresponding network node 16. Note that although only two UEs 22 and three network nodes 16 are shown for convenience, the communication system may include many more UEs 22 and network nodes 16.

Also, it is contemplated that a UE 22 can be in simultaneous communication and/or configured to separately communicate with more than one network node 16 and more than one type of network node 16. For example, a UE 22 can have dual connectivity with a network node 16 that supports LTE and the same or a different network node 16 that supports NR. As an example, UE 22 can be in communication with an eNB for LTE/E-UTRAN and a gNB for NR/NG-RAN.

A network node 16 is configured to include a NW Al unit 32 which is configured to support autoencoder-based AI/ML model selection based on drift detection. A UE 22 is configured to include a UE Al unit 34 which is configured to support autoencoder-based AI/ML model selection based on drift detection.

Example implementations, in accordance with an embodiment, of the UE 22 and network node 16 discussed in the preceding paragraphs will now be described with reference to FIG. 6. The communication system 10 includes a network node 16 provided in a communication system 10 and including hardware 58 enabling it to communicate with the UE 22. The hardware 58 may include a communication interface 60 for setting up and maintaining a wired or wireless connection with an interface of a different communication device of the communication system 10, as well as a radio interface 62 for setting up and maintaining at least a wireless connection 64 with a UE 22 located in a coverage area 18 served by the network node 16. The radio interface 62 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers.

In the embodiment shown, the hardware 58 of the network node 16 further includes processing circuitry 68. The processing circuitry 68 may include a processor 70 and a memory 72. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 68 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 70 may be configured to access (e.g., write to and/or read from) the memory 72, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory).

Thus, the network node 16 further has software 74 stored internally in, for example, memory 72, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the network node 16 via an external connection. The software 74 may be executable by the processing circuitry 68. The processing circuitry 68 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by network node 16. Processor 70 corresponds to one or more processors 70 for performing network node 16 functions described herein. The memory 72 is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 74 may include instructions that, when executed by the processor 70 and/or processing circuitry 68, causes the processor 70 and/or processing circuitry 68 to perform the processes described herein with respect to network node 16. For example, processing circuitry 68 of the network node 16 may include NW Al unit 32 configured to support autoencoder-based AI/ML model selection based on drift detection. The communication system 10 further includes the UE 22 already referred to. The UE 22 may have hardware 80 that may include a radio interface 82 configured to set up and maintain a wireless connection 64 with a network node 16 serving a coverage area 18 in which the UE 22 is currently located. The radio interface 82 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers.

The hardware 80 of the UE 22 further includes processing circuitry 84. The processing circuitry 84 may include a processor 86 and memory 88. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 84 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 86 may be configured to access (e.g., write to and/or read from) memory 88, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory).

Thus, the UE 22 may further comprise software 90, which is stored in, for example, memory 88 at the UE 22, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the UE 22. The software 90 may be executable by the processing circuitry 84. The software 90 may include a client application 92. The client application 92 may be operable to provide a service to a human or non-human user via the UE 22. The client application 92 may interact with the user to generate the user data that it provides.

The processing circuitry 84 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by UE 22. The processor 86 corresponds to one or more processors 86 for performing UE 22 functions described herein. The UE 22 includes memory 88 that is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 90 and/or the client application 92 may include instructions that, when executed by the processor 86 and/or processing circuitry 84, causes the processor 86 and/or processing circuitry 84 to perform the processes described herein with respect to UE 22. For example, the processing circuitry 84 of the UE 22 may include a UE Al unit 34 configured to configured to support autoencoder-based AI/ML model selection based on drift detection.

In some embodiments, the inner workings of the network node 16 and UE 22may be as shown in FIG. 6 and independently, the surrounding network topology may be that of FIG. 5.

The wireless connection 64 between the UE 22 and the network node 16 is in accordance with the teachings of the embodiments described throughout this disclosure. More precisely, the teachings of some of these embodiments may improve the data rate, latency, and/or power consumption and thereby provide benefits such as reduced user waiting time, relaxed restriction on file size, better responsiveness, extended battery lifetime, etc.

In some embodiments, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve.

Although FIGS. 5 and 6 show various “units” such as NW Al unit 32, and UE Al unit 34 as being within a respective processor, it is contemplated that these units may be implemented such that a portion of the unit is stored in a corresponding memory within the processing circuitry. In other words, the units may be implemented in hardware or in a combination of hardware and software within the processing circuitry.

FIG. 7 is a flowchart of an example process in a UE 22 according to some embodiments of the present disclosure for supporting autoencoder-based AI/ML model selection based on drift detection. One or more blocks described herein may be performed by one or more elements of UE 22 such as by one or more of processing circuitry 84 (including the UE Al unit 34), processor 86, radio interface 82 and/or communication interface 60. UE 22 is configured to determine (Block SI 00) a first model monitoring metric for a first model of a plurality of auto-encoder models using at least a first encoder and a first decoder associated with the first model. UE 22 is configured to, optionally, switch (Block SI 02) to a second model of the plurality of auto-encoder models based on the first model monitoring metric exceeding a threshold drift value.

In some embodiments, the first model monitoring metric for the first model is determined by receiving and/or storing the first encoder and the first decoder associated with the first model, providing a first radio measurement input to the first encoder to generate a first location output, providing the first location output to the first decoder to generate a first estimated radio measurement input, and comparing the first estimated radio measurement input with the first radio measurement input. In some embodiments, the UE 22 is further configured to store the plurality of auto-encoder models.

FIG. 8 is a flowchart of another example process in a UE 22 according to some embodiments of the present disclosure for supporting autoencoder-based AI/ML model selection based on drift detection. One or more blocks described herein may be performed by one or more elements of UE 22 such as by one or more of processing circuitry 84 (including the UE Al unit 34), processor 86, radio interface 82 and/or communication interface 60. UE 22 is configured to determine (Block SI 04) a first model monitoring metric for a first model of a plurality of auto-encoder models. UE is configured to determine (Block SI 06) a second model monitoring metric for a second model of the plurality of auto-encoder models. UE is configured to select (Block SI 08) the first or second model based on the first and second model monitoring metrics.

FIG. 9 is a flowchart of an example process in a network node 16 for supporting autoencoder-based AI/ML model selection based on drift detection. One or more blocks described herein may be performed by one or more elements of network node 16 such as by one or more of processing circuitry 68 (including the NW Al unit 32), processor 70, radio interface 62 and/or communication interface 60. Network node 16 is configured to determine (Block SI 10) a first model monitoring metric for a first model of a plurality of auto-encoder models using at least a first encoder and a first decoder associated with the first model. Network node 16 is configured to, optionally, switch (Block SI 12) to a second model of the plurality of auto-encoder models based on the first model monitoring metric exceeding a threshold drift value.

In some embodiments, the first model monitoring metric for the first model is determined by receiving and/or storing the first encoder and the first decoder associated with the first model, providing a first radio measurement input to the first encoder to generate a first location output, providing the first location output to the first decoder to generate a first estimated radio measurement input, and comparing the first estimated radio measurement input with the first radio measurement input. In some embodiments, the network node 16 is further configured to store the plurality of auto-encoder models.

FIG. 10 is a flowchart of an example process in a network node 16 for supporting autoencoder-based AI/ML model selection based on drift detection. One or more blocks described herein may be performed by one or more elements of network node 16 such as by one or more of processing circuitry 68 (including the NW Al unit 32), processor 70, radio interface 62 and/or communication interface 60. Network node 16 is configured to determine (Block SI 14) a first model monitoring metric for a first model of a plurality of auto-encoder models. Network node 16 is configured to determine (Block SI 16) a second model monitoring metric for a second model of the plurality of auto-encoder models. Network node is configured to select (Block SI 18) the first or second model based on the first and second model monitoring metrics.

FIG. 11 is a flowchart of yet another example process in a UE 22 according to some embodiments of the present disclosure for supporting autoencoder-based AI/ML model selection based on drift detection. One or more blocks described herein may be performed by one or more elements of UE 22 such as by one or more of processing circuitry 84 (including the UE Al unit 34), processor 86, radio interface 82 and/or communication interface 60. UE 22 is configured to determine (Block S200) a monitoring metric for each one of a plurality of machine learning, ML, models. UE 22 is configured to determine (Block S202) based on the monitoring metrics, whether to implement one of the plurality of ML models. UE 22 is configured to communicate (Block S204) with the network node 16 based on the determination of whether to implement one of the plurality of ML models.

In some embodiments, the determining whether to implement one of the plurality of ML models comprises determining not to implement any of the plurality of ML models.

In some embodiments, the determining whether to implement one of the plurality of ML models comprises selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

In some embodiments, determining whether to implement one of the plurality of ML models comprises comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

In some embodiments, the UE 22 is further configured to detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

FIG. 12 is a flowchart of an example process in a network node 16 for supporting autoencoder-based AI/ML model selection based on drift detection. One or more blocks described herein may be performed by one or more elements of network node 16 such as by one or more of processing circuitry 68 (including the NW Al unit 32), processor 70, radio interface 62 and/or communication interface 60. Network node 16 is configured to determine (Block S206) a monitoring metric for each one of a plurality of machine learning, ML, models. Network node 16 is configured to determine (Block S208) , based on the monitoring metrics, whether to implement one of the plurality of ML models. Network node 16 is configured to communicate (Block S210) with the UE 22 based on the determination of whether to implement one of the plurality of ML models.

In some embodiments the determining whether to implement one of the plurality of ML models comprises determining not to implement any of the plurality of ML models.

In some embodiments the determining whether to implement one of the plurality of ML models comprises selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

In some embodiments determining whether to implement one of the plurality of ML models comprises comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

In some embodiments, network node 16 is further configured to detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

Having described the general process flow of arrangements of the disclosure and having provided examples of hardware and software arrangements for implementing the processes and functions of the disclosure, the sections below provide details and examples of arrangements for supporting autoencoder-based AI/ML model selection based on drift detection.

Embodiments of the present disclosure may support AI/ML model management methods using model monitoring mechanisms, as described herein. As an illustrative example, a use case of AI/ML based positioning is provided. While positioning is a typical use case of AI/ML for wireless communication, this is a non-limiting example and is used to illustrate the methodology only. It is to be understood by those skilled in the art that the methodologies and design principles disclosed herein may be extended to a variety of other use cases beyond UE positioning.

It is also noted that the disclosed methodologies and design principles may be applied to AI/ML models deployed in various entities in a wireless communications network, including but not limited to a UE 22 and/or a network node 16.

Pretraining multiple Al models and especially forming auto-encoder-like inference units

In describing some embodiments of the present disclosure, it may be assumed that multiple AI/ML models may in some cases be available for the UE 22 (and/ or network node 16) to select on demand. For example, if a UE 22 is expected to encounter multiple different deployment scenarios and/or configurations, then multiple AI/ML models may be pre-trained and stored in (and/or signaled to) the UE 22, where each AI/ML model is trained for a targeted deployment scenario and/or configuration.

In some embodiments, when an AI/ML model g(.) is trained to map input X to output h to fulfill a desired inference functionality, it may also be considered as an Encoder part of an autoencoder. Thus, a corresponding Decoder f(.) may be trained to map h back to an estimation X, where X approximates X as closely as possible. The same training dataset may be used to train the Encoder part and the Decoder part, for example, where the model input and model output relationship is reversed from encoder to decoder. When the AI/ML model g(.) (which is also the Encoder part of the autoencoder) is deployed in the field, the corresponding Decoder f(.) can be used for model monitoring and model selection of the AI/ML model g(.). For any of the embodiments described herein, one or more of the AI/ML model(s), the Encoder(s), and/or Decoder(s) may be stored in, implemented in, and/or configured to run on the processing circuitry 84 of UE 22, including the UE Al Unit 34, and/or in the processing circuitry 68 of network node 16, including the NW Al Unit 32.

In some embodiments, model monitoring and model selection may be performed as follows. Let n Al models {goQ, gi(.), . . . gn-i(.)} and their corresponding {foQ, fi(.), . . . fn-i(.)} be accessible by the UE 22. When data for model monitoring is collected for the current deployment scenario and/or configuration, the UE 22 (and/or network node 16) may be configured to generate model monitoring metric Mj for each of model gj(.) using fj(.), j=0, 1, . . . , n-1. The model g(.) with the best model monitoring metric is then selected for the current deployment scenario and/or configuration. In some embodiments, the data for model monitoring may be that of model input X only, i.e., without ground truth labels which are true values of model output h. Thus, with this method, model monitoring and selection may be accomplished without requiring labelled data.

Example use case ofUE 22 Al positioning: Al Encoder

In some embodiments, such as for UE 22 positioning, an AI/ML model g(.) may be trained to obtain model output h from the model input X of radio signal measurement, where the radio signal measurement can be a RSRP tensor, or a channel impulse response tensor, or a power delay profile tensor, or other desired radio signal measurements. In some embodiments, the training may occur at a UE 22, network node 16, or other entity, and may occur “offline”, for example, and/or may be trained using “live” data. For AI/ML based positioning, the model output h may be position coordinates [x, y, z] of the target UE 22, and/or certain metrics (e.g., a vector of time-of-arrival (ToA)) that may be used to calculate the position coordinates of the target UE 22. The AI/ML model g(.) may also be considered an Encoder part of a positioning autoencoder.

Example use case ofUE 22 Al positioning: Al Decoder

In some embodiments, for an AI/ML model g(.) (also the Encoder part of a positioning autoencoder), a corresponding Decoder f(.) may be trained as well, using the same training dataset (only the model input - model output relationship is reversed). In some embodiments, the training may occur at a UE 22, network node 16, or other entity, and may occur “offline”, for example, and/or may be trained using “live” data. The Decoder f(.) maps h back to an estimation of the radio signal measurement X. For each candidate AI/ML model gj(.) available to the UE 22 for positioning, a corresponding fj(.) is provided, j=0, 1, ..., n-1. When the AI/ML model g(.) is deployed in the field to support UE 22 positioning, its corresponding Decoder f(.) can be used for model monitoring and model selection of the AI/ML model g(.).

Examples in which the UE 22 collects online data and tests the data with a pretrained auto-encoder unit

In some embodiments, considering a property of auto-encoders, the distance calculation may be according to: UE 22 and/or network node 16 may be configured to determine the distance according to the above calculation, which may further reflect a certain format of sum of the element-by- element comparison of the two length-L vectors, X_t and X_t, since ideally X_t = X_t element- by-element, such as a sum of squared errors.

If the model (e.g., as deployed on a UE 22 and/or network node 16) is working well for the deployed environment, then a distance metric at time t between the encoder input X_t and decoder output X_t is expected to be small. The distance metric at model deployment, if working properly may be comparable to the metrics at training phase.

In some embodiments, the distance calculation may be performed (e.g., by UE 22 and/or network node 16) in one or more ways, such as:

• One example metric is d_t = statistical mean of the Euclidean distance between X_t, X_t. If it is chosen (e.g., by UE 22 and/or network node 16, and/or based on preconfigured information) for monitoring the drift, it may be within a small difference when compared to the reference value at the training phase. If the mean at operation is larger than the mean at training by a threshold value, that result may indicate a possible model drift.

• Another example metric is d_t = p-norm distance between the two length-L vectors X_t,X_t

When p = 1, the distance is the taxicab norm,

When p = 2, the distance is the Euclidean norm, d_t =

When p = oo, the distance is the maximum norm, d_t =

In a model monitoring time window from time t₀ to t₀ + T — 1, a series of d_t may be obtained (e.g., by UE 22 and/or network node 16) as follows: {d_to, d_to+1, d_to+T-1}. Then the statistical distribution of {d_to, d_to+1, ... , d_to+T-1] may be compared to the statistical distribution of the corresponding set of distances obtained using training dataset.

In general, if the autoencoder continues to work well in the deployed environment, then d_t is expected to be small (almost) all of the time, and its distribution matches that at training, indicating that decoder output X_t is able to closely match the encoder input X_t.

Examples where UE selects one model out of multiple candidate AI/ML models according to the input and output dataset distance

In some embodiments, multiple candidates of auto-encoder models may be accessible by the deployed UE 22 (and/or network node 16), for example, pre-stored on the UE 22, and/or sent from one UE 22 to another UE 22, and/or provided by a network node 16 to the UE 22, and/or retrieved over-the-top via downloading from a model storage entity. Here, the model storage entity may be located on the network side (e.g., at network node 16, or on the UE 22 side, or at a neutral side.

In some embodiments, during operation, UE 22 (and/or network node 16) is configured to choose one or multiple of the Al models g(.) and conduct model drift detection. The UE 22 selects the best Al model g(.), for example, the AI/ML model gj(.) with the smallest model monitoring metric is considered the best. Other criteria may be used to determine the best Al model(s) g(.).

Example criteria candidates for selections

In some embodiments, the UE 22 (and/or network node 16) is configured to calculate the distance metric d_t between auto-encoder inputs and output data sets X_t and X_t for each of the n autoencoders, where the j -th autoencoder is composed of Encoder gj(.) and Decoder fj(.). Several criteria may be utilized for model selection, including one or more of:

• Criteria (a). Among the n available models g(.), the UE 22 (and/or network node 16) is configured to select the model gk(.) with the best metric. In some embodiments, the best metric corresponds to the smallest model monitoring metric, for example, the smallest distance metric d_t.

• Criteria (b). For the model selection, a selection threshold is predefined (e.g., according to configuration information stored in UE 22 memory 72). Thus, in some embodiments, a model gk(.) may need to have a model monitoring metric smaller than the threshold, before gk(.) may be considered an appropriate model for the current scenario and/or configuration. When using Criteria (b) by itself, zero, one, or more than one models g(.) may satisfy the criteria, and further actions may be necessary (e.g., by UE 22 and/or network node 16) to make the model selection decision.

In some embodiments, the UE 22 (and/or network node 16) may be configured to use either Criteria (a), or Criteria (b), or a combination of (a) and (b). When using Criteria (a) and (b) in combination, a model gk(.) may be selected if it has the best model monitoring metric among all candidates g(.), and also if its model monitoring metric is smaller than the threshold.

In the above description, the discussion assumes that the best metric corresponds to the smallest model monitoring metric, for example, the smallest distance metric d_t. Embodiments of the present disclosure are not so limited, and other types of metrics may be defined and utilized as well. For example, depending on the definition of the metric, the best metric may correspond to the largest metric value, for example, when the metric is calculated on the similarity between X_t and X_t.

Example model monitoring based model selection management

In some embodiments, UE 22 (and/or network node 16) may be configured to switch the selected AI/ML model without performing a further model update or finetuning, e.g., if the selection criterion is satisfied. In another method, the selected AI/ML model with the best metric is further updated or fine-tuned before actual deployment.

While operating, UE 22 (and/or network node 16) may be configured to perform model monitoring continuously, or periodically, or when triggered. When a model drift alarm is sounded, the UE 22 (and/or network node 16) may be configured to prepare to switch to another model, where the preparation stage may include the model selection step.

Example UE 22 Al model selection : general flow chat of operational steps

As in illustrated in the process flow diagram of FIG. 13, in some embodiments, before operation or while in operation, multiple models may be trained (Step S300) (e.g., by UE 22, by network node 16, by another network entity, in a laboratory setting, etc.), instead of a single model, and/or such models continuously updated and maintained, so that multiple candidate models may be available to UE 22 (and/or network node 16) for model selection. In some embodiments, the models include autoencoder units. UE 22 and/or network node 16 is configured to initiate a monitoring instance (Step S302).

In some embodiments, during model deployment, one or more phases may be configured in UE 22 and/or network node 16, including:

• model drift detection (MDD), which may also be referred to as model monitoring (Step S304), e.g., as performed by UE 22 and/or network node 16;

• model selection (Step S306), which may include calculating (e.g., by UE 22 and/or network node 16) model monitoring metrics for multiple candidate models, and selecting the best model (if available, e.g., if any of the candidate models exceed a minimum performance/accuracy threshold at Step S308) for the current scenario and/or configuration and/or environment (e.g., as experienced by UE 22 and/or network node 16); • (optional) If the metric exceeds a minimum threshold, at Step S310, UE 22 and/or network node 16 may be configured to perform model tuning or model fine-tuning, e.g., based on the model drift detection information or alarm;

• model switching (Step S312), which may use one or more methods described herein to deactivate the current model and activate the best model (if a best model is identified, if the threshold is not satisfied in Step S308, etc.).

Possible data-driven model deactivation and a fallback to knowledge-driven model In the example of FIG. 13 and the discussion above, it may in some cases be assumed that a proper model may be found among the candidate models for the current scenario and/or configuration, or a best candidate model may be selected and further finetuned. In practice, however, it may be possible that none of the candidate models are proper (e.g., no candidate models satisfy the model monitoring threshold), and model (fine-)tuning is not feasible (e.g., the model has drifted too far, and/or (fine-)tuning is precluded by implementation limitations). In this case, the UE 22 (and/or network node 16) may be configured to decide to switch to a fallback mode, for example, deactivating the AI/ML model(s) for the given functionality and may instead use a conventional non- AI/ML method.

Example model monitoring based model selection

In some embodiments, a practical model selection is supported, which may integrate model drift detection and model switching. This may beneficially mitigate the generalization issues and thereby improve model accuracy.

In at least some embodiments, an autoencoder structure is utilized, which may eliminate or reduce the need for UE 22 to utilize accurate label data for model monitoring and model selection, which enables these techniques to be applied to a variety of real world scenarios where accurate label data is practically unavailable.

As noted herein, although embodiments may be described with respect to a UE 22- side model as an example, it is to be understood by those skilled in the art that the same methodologies and procedures may be applicable to AI/ML models deployed in other devices or nodes, for example, network nodes 16 such as gNB, LMF, etc. Example Embodiments:

Example Al . A UE 22 configured to communicate with a network node, the UE 22 configured to, and/or comprising a radio interface and/or processing circuitry configured to: determine a first model monitoring metric for a first model of a plurality of autoencoder models using at least a first encoder and a first decoder associated with the first model; and optionally, switch to a second model of the plurality of auto-encoder models based on the first model monitoring metric exceeding a threshold drift value.

Example A2. The UE 22 of Example Al, wherein the first model monitoring metric for the first model is determined by: receiving and/or storing the first encoder and the first decoder associated with the first model; providing a first radio measurement input to the first encoder to generate a first location output; providing the first location output to the first decoder to generate a first estimated radio measurement input; and comparing the first estimated radio measurement input with the first radio measurement input.

Example A3. The UE 22 of any one of Examples Al and A2, wherein the UE is further configured to store the plurality of auto-encoder models.

Example A4. A UE 22 configured to communicate with a network node, the UE 22 configured to, and/or comprising a radio interface and/or processing circuitry configured to: determine a first model monitoring metric for a first model of a plurality of autoencoder models; determine a second model monitoring metric for a second model of the plurality of auto-encoder models; and select the first or second model based on the first and second model monitoring metrics.

Example Bl. A method implemented in a UE 22 configured to communicate with a network node, the method comprising: determining a first model monitoring metric for a first model of a plurality of autoencoder models using at least a first encoder and a first decoder associated with the first model; and optionally, switching to a second model of the plurality of auto-encoder models based on the first model monitoring metric exceeding a threshold drift value. Example B2. The method of Example Bl, wherein the first model monitoring metric for the first model is determined by: receiving and/or storing the first encoder and the first decoder associated with the first model; providing a first radio measurement input to the first encoder to generate a first location output; providing the first location output to the first decoder to generate a first estimated radio measurement input; and comparing the first estimated radio measurement input with the first radio measurement input.

Example B3. The method of any one of Examples Bl and B2, wherein the method further comprises storing the plurality of auto-encoder models.

Example B4. A method implemented in a UE 22 configured to communicate with a network node, the method comprising: determining a first model monitoring metric for a first model of a plurality of autoencoder models; determining a second model monitoring metric for a second model of the plurality of auto-encoder models; and selecting the first or second model based on the first and second model monitoring metrics.

Example Cl . A network node 16 configured to communicate with a UE 22, the network node 16 configured to, and/or comprising a radio interface and/or processing circuitry configured to: determine a first model monitoring metric for a first model of a plurality of autoencoder models using at least a first encoder and a first decoder associated with the first model; and optionally, switch to a second model of the plurality of auto-encoder models based on the first model monitoring metric exceeding a threshold drift value.

Example C2. The network node 16 of Example Cl, wherein the first model monitoring metric for the first model is determined by: receiving and/or storing the first encoder and the first decoder associated with the first model; providing a first radio measurement input to the first encoder to generate a first location output; providing the first location output to the first decoder to generate a first estimated radio measurement input; and comparing the first estimated radio measurement input with the first radio measurement input.

Example C3. The network node 16 of any one of Examples Cl and C2, wherein the network node is further configured to store the plurality of auto-encoder models.

Example C4. A network node 16 configured to communicate with a UE 22, the network node 16 configured to, and/or comprising a radio interface and/or processing circuitry configured to: determine a first model monitoring metric for a first model of a plurality of autoencoder models; determine a second model monitoring metric for a second model of the plurality of auto-encoder models; and select the first or second model based on the first and second model monitoring metrics.

Example DI . A method implemented in a network node 16 configured to communicate with a UE 22, the method comprising: determining a first model monitoring metric for a first model of a plurality of autoencoder models using at least a first encoder and a first decoder associated with the first model; and optionally, switching to a second model of the plurality of auto-encoder models based on the first model monitoring metric exceeding a threshold drift value.

Example D2. The method of Example DI, wherein the first model monitoring metric for the first model is determined by: receiving and/or storing the first encoder and the first decoder associated with the first model; providing a first radio measurement input to the first encoder to generate a first location output; providing the first location output to the first decoder to generate a first estimated radio measurement input; and comparing the first estimated radio measurement input with the first radio measurement input.

Example D3. The method of any one of Examples DI and D2, wherein the method further comprises storing the plurality of auto-encoder models. Example D4. A method implemented in a network node 16 configured to communicate with a UE 22, the method comprising: determining a first model monitoring metric for a first model of a plurality of autoencoder models; determining a second model monitoring metric for a second model of the plurality of auto-encoder models; and selecting the first or second model based on the first and second model monitoring metrics.

As will be appreciated by one of skill in the art, the concepts described herein may be embodied as a method, data processing system, computer program product and/or computer storage media storing an executable computer program. Accordingly, the concepts described herein may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or “module.” Any process, step, action and/or functionality described herein may be performed by, and/or associated to, a corresponding module, which may be implemented in software and/or firmware and/or hardware. Furthermore, the disclosure may take the form of a computer program product on a tangible computer usable storage medium having computer program code embodied in the medium that can be executed by a computer. Any suitable tangible computer readable medium may be utilized including hard disks, CD-ROMs, electronic storage devices, optical storage devices, or magnetic storage devices.

Some embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, systems and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer (to thereby create a special purpose computer), special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable memory or storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

It is to be understood that the functions/acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

Computer program code for carrying out operations of the concepts described herein may be written in an object oriented programming language such as Python, Java® or C++. However, the computer program code for carrying out operations of the disclosure may also be written in conventional procedural programming languages, such as the "C" programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, all embodiments can be combined in any way and/or combination, and the present specification, including the drawings, shall be construed to constitute a complete written description of all combinations and subcombinations of the embodiments described herein, and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.

Abbreviations that may be used in the preceding description include:

Al Artificial Intelligence

BS Base Station

CE Channel Estimate

MAC Medium Access Control

ML Machine Learning

PBCH Physical Broadcast Channel

PSS Primary Synchronization Signal

RRC Radio Resource Control

RF Radio Frequency

SSS Secondary Synchronization Signal

TRX Transceiver

UE User Equipment

It will be appreciated by persons skilled in the art that the embodiments described herein are not limited to what has been particularly shown and described herein above. In addition, unless mention was made above to the contrary, it should be noted that all of the accompanying drawings are not to scale. A variety of modifications and variations are possible in light of the above teachings without departing from the scope of the following claims.

Claims

What is claimed is:

1. A user equipment, UE, (22) configured to communicate with a network node 16, the UE (22) comprising processing circuitry (84) configured to: determine a monitoring metric for each one of a plurality of machine learning, ML, models; determine, based on the monitoring metrics, whether to implement one of the plurality of ML models; communicate with the network node (16) based on the determination of whether to implement one of the plurality of ML models.

2. The UE (22) of Claim 1, wherein the determining whether to implement one of the plurality of ML models comprises determining not to implement any of the plurality of ML models.

3. The UE (22) of Claim 1, wherein the determining whether to implement one of the plurality of ML models comprises selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

4. The UE (22) of any one of Claims 1-3, wherein determining whether to implement one of the plurality of ML models comprises comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of: determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

5. The UE (22) of any one of Claims 1-4, wherein the processing circuitry (84) is further configured to: detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

6. A method performed by a user equipment, UE, (22) configured to communicate with a network node 16, the method comprising: determining (S200) a monitoring metric for each one of a plurality of machine learning, ML, models; determining (S202) , based on the monitoring metrics, whether to implement one of the plurality of ML models; communicating (S204) with the network node (16) based on the determination of whether to implement one of the plurality of ML models.

7. The method of Claim 6, wherein the determining whether to implement one of the plurality of ML models comprises determining not to implement any of the plurality of ML models.

8. The method of Claim 6, wherein the determining whether to implement one of the plurality of ML models comprises selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

9. The method of any one of Claims 6-8, wherein determining whether to implement one of the plurality of ML models comprises comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

10. The method of any one of Claims 6-9, further comprising: detecting that the implemented one of the plurality of ML models has drifted; and determining whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

11. A network node 16 configured to communicate with a user equipment, UE, (22) the network node 16 comprising processing circuitry (68) configured to: determine a monitoring metric for each one of a plurality of machine learning, ML, models; determine, based on the monitoring metrics, whether to implement one of the plurality of ML models; communicate with the UE (22) based on the determination of whether to implement one of the plurality of ML models.

12. The network node (16) of Claim 11, wherein the determining whether to implement one of the plurality of ML models comprises determining not to implement any of the plurality of ML models.

13. The network node (16) of Claim 11, wherein the determining whether to implement one of the plurality of ML models comprises selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

14. The network node (16) of any one of Claims 11-13, wherein determining whether to implement one of the plurality of ML models comprises comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of: determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

15. The network node (16) of any one of Claims 11-14, wherein the processing circuitry (68) is further configured to: detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.

16. A method performed by a network node (16) configured to communicate with a user equipment, UE, (22) method comprising: determining (S206) a monitoring metric for each one of a plurality of machine learning, ML, models; determining (S208), based on the monitoring metrics, whether to implement one of the plurality of ML models; communicating (S210) with the UE (22) based on the determination of whether to implement one of the plurality of ML models.

17. The method of Claim (16), wherein the determining whether to implement one of the plurality of ML models comprises determining not to implement any of the plurality of ML models.

18. The method of Claim (16), wherein the determining whether to implement one of the plurality of ML models comprises selecting one of the plurality of ML models based on a comparison of the monitoring metrics of the plurality of ML models, and implementing the selected ML model.

19. The method of any one of Claims 16-18, wherein determining whether to implement one of the plurality of ML models comprises comparing the monitoring metric of each of the plurality of ML models to a threshold, and one of: determining not to implement any of the plurality of ML models based on the monitoring metrics of the plurality of ML models all being lower than the threshold; and implementing one of the plurality of ML models whose monitoring metric is greater than or equal to the threshold.

20. The method of any one of Claims 16-19, further comprising: detect that the implemented one of the plurality of ML models has drifted; and determine whether to implement a different one of the plurality of ML models based on the determination that the implemented one of the plurality of ML models has drifted.