WO2023012351A1

WO2023012351A1 - Controlling and ensuring uncertainty reporting from ml models

Info

Publication number: WO2023012351A1
Application number: PCT/EP2022/072135
Authority: WO
Inventors: Henrik RYDÉN; Philipp BRUHN; Angelo Centonza; Luca LUNARDI; German BASSI
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2021-08-05
Filing date: 2022-08-05
Publication date: 2023-02-09
Also published as: EP4381707A1

Abstract

Systems and methods related to controlling and ensuring uncertainty reporting from Machine Learning (ML) models are disclosed. In one embodiment, a method performed by a first node comprises sending, to a second node, one or more output predictions from a first ML model together with uncertainty information about the one or more output predictions. In this manner, the robustness and/or consistency of models and/or procedures that use the one or more output predictions can be improved. Corresponding embodiments of a first node are also disclosed. Embodiments of a second node and methods of operation thereof are also disclosed.

Description

CONTROLLING AND ENSURING UNCERTAINTY REPORTING FROM ML MODELS

This application claims the benefit of provisional patent application serial number 63/229,675, filed August 5, 2021, the disclosure of which is hereby incorporated herein by reference in its entirety.

Technical Field

The present disclosure relates to Machine Learning (ML) and Artificial Intelligence (Al) and more specifically relates to providing output predictions from a ML model at a training/inference node to an actor node, particularly in a manner that is suitable for use in a Radio Access Network (RAN) of a cellular communications system.

Problems to be addressed by the 3^rd Generation Partnership Project (3GPP) Radio Access Networks (RAN) (as discussed at RAN3 meeting #112e, details available at https://www.3gpp.org/ftp/TSG_RAN/WG3_lu/TSGR3_112- e/Docs/ as of 4 August 2021) include:

• If the inference function provides output predictions, an optional indication of the accuracy level for which the inference model is trained should be indicated to the nodes that request/subscribe to this information;

• the feasibility of a "validity time” (i.e., "best before” for the prediction result) as additional information provided by the Model Inference function together with the Inference output;

• Output from one model as input to another; and

• High-level principles for inference function.

To increase the awareness of the traffic dynamics and enable more improved traffic steering decisions, it is possible to complement load measurements currently exposed over RAN interfaces with information related to measured and predicted traffic from User Equipments (UEs) as well as predicted load from neighboring RAN nodes. For example:

• A RAN node may request and obtain UE traffic measurements and predictions, based on the real end-user behavior. Such measurements and predictions may provide RAN with insights on the UE traffic. The mechanism would require configuring the UE to collect and report such measurements and predictions over the Radio Resource Control (RRC) protocol.

• A RAN node may predict its own load as well as the load of a neighboring RAN node. Predictions may be achieved by considering the own load, load information received from neighbor RAN nodes, and traffic predictions from UEs. Finally, RAN predictions may be signaled between RAN nodes.

Chained Models

An issue under consideration is that of chained Machine Learning (ML) models, where the output from one model is used as the input for another model. An example of chained ML models is shown in Figure 1 , in which two chained ML models, namely, ML model 1 and ML model 2 are shown. The two chained ML models are hosted in different nodes, for example, base stations such as different New Radio (NR) next generation Node Bs (gNBs) or components thereof (e.g., gNB Central Unit (CU) Control Planes (CPs) (i.e., CU-CPs), or gNB Distributed Units (DUs), as shown Figure 1.

Uncertainties in Predictions

Uncertainty is a key notion in Artificial Intelligence (Al) and Machine Learning (ML), and uncertainty quantification is a key element for trustworthy and explainable Al and ML. Accuracy quantifies how close a prediction is to the true value and can only be measured once the true value is known. It is often determined as average for many predictions and used to evaluate and compare the predictive power of Al and ML algorithms. Uncertainty assesses how much a prediction may differ from the true value and can be estimated along with the prediction.

There are two inherently different sources of uncertainty, often referred to as aleatoric and epistemic uncertainty. Aleatoric (or statistical) uncertainty refers to the noise in the data, meaning the probabilistic variability of the output due to inherent random effects. It is irreducible, which means that it cannot be reduced by providing more data or choosing a different Al or ML model or algorithm. By contrast, epistemic (or systematic) uncertainty comes from limited data and knowledge about the system and underlying processes and phenomena. Regarding Al and ML, it refers to the lack of knowledge about the perfect model (for example, lack of knowledge of best model parameters), typically due to inappropriate or insufficient training data. The epistemic uncertainty part of the total uncertainty is in principle reducible, for example by providing more data.

Epistemic uncertainty describes what the model does not know because it was unable to learn from the training data. Epistemic uncertainty therefore results from limited data and knowledge. Given enough training samples, epistemic uncertainty will decrease. Epistemic uncertainty can arise in areas where there are fewer samples for training. In principle, it can also occur when the model is too simple to capture the complexity of the data, for instance, when a linear model is fitted to or trained with nonlinear data.

Aleatoric uncertainty is the uncertainty arising from the natural stochasticity of observations. Aleatoric uncertainty cannot be reduced even when more data is provided. Aleatoric uncertainty can further be categorized into homoscedastic uncertainty, uncertainty which stays constant for different inputs, and heteroscedastic uncertainty. Heteroscedastic uncertainty depends on the inputs to the model, with some inputs potentially having more noisy outputs than others.

Figure 2 illustrates examples of aleatoric and epistemic uncertainties.

When training a ML model, the selected algorithm will affect the level of uncertainty that can be estimated. In general, a more complex model, such as a Gaussian process, can estimate the uncertainty more accurately than a simpler model. An example of uncertainty estimation using different models is shown in Figure 3. Figure 3 illustrates the 95% prediction intervals for two models obtained by different algorithms. The first model is derived from a Gaussian process, which can learn the heteroscedastic noise process comprised in the training data. Thus, the size of the prediction interval for this model depends on the input to the model. Figure 3 shows that the Gaussian process estimates a higher (epistemic) uncertainty for 2<x<4, due to the lack of training samples, and a higher (aleatoric) uncertainty for 6<x<8, due to the higher noise level. By contrast, the second model is obtained by linear regression with polynomial features, which is a much simpler method. In this case, the size of the prediction interval is constant for all inputs, so it does not depend on the input. Therefore, the uncertainty estimation is much less accurate, since it cannot capture the heteroscedastic noise process in a meaningful way.

Summary

Systems and methods related to controlling and ensuring uncertainty reporting from Machine Learning (ML) models are disclosed. In one embodiment, a method performed by a first node comprises sending, to a second node, one or more output predictions from a first ML model together with uncertainty information about the one or more output predictions. In this manner, the robustness and/or consistency of models and/or procedures that use the one or more output predictions can be improved.

In one embodiment, the uncertainty information comprises information that indicates one or more types of uncertainty the one or more output predictions are subject to. In one embodiment, the uncertainty information further comprises information about the one or more types of uncertainty the one or more output predictions are subject to.

In one embodiment, the uncertainty information comprises information that indicates whether the uncertainty information is per-sample uncertainty during inference, information that indicates whether the uncertainty information is statistical uncertainty from training of the first ML model, information that indicates whether the uncertainty information comprises epistemic uncertainty in inference, or any combination thereof.

In one embodiment, the uncertainty information comprises, per-sample uncertainty information for each output prediction from the first ML model, the per-sample uncertainty information being dependent on respective inputs to the first ML model.

In one embodiment, an output prediction from the first model is a point estimate, and the uncertainty information comprises an uncertainty value for the point estimate.

In one embodiment, the uncertainty information comprises a probabilistic prediction for a respective output prediction, and the output prediction can be derived from the probabilistic prediction.

In one embodiment, the uncertainty information comprises information that indicates that the uncertainty information comprises information about a statistical uncertainty of the one or more output predictions as a result of training of the first ML model. In one embodiment, the information about the statistical uncertainty of the one or more output predictions as a result of training of the first ML model comprises information about a homoscedastic uncertainty of the one or more output predictions. In one embodiment, the information about the statistical uncertainty of the one or more output predictions as a result of training of the first ML model comprises information about a heteroscedastic uncertainty of the one or more output predictions.

In one embodiment, the method further comprises receiving, from the second node, information that indicates one or more ML model uncertainty requirements. In one embodiment, the method further comprises determining whether or not to initiate the sending of the one or more output predictions from the first ML model to the second node, based on the ML model uncertainty requirements received from the second node. Sending the one or more output predictions together with the uncertainty information about the one or more output predictions comprises sending the one or more output predictions together with the uncertainty information about the one or more output predictions to the second node in response to determining to initiate the sending of the one or more output predictions from the first ML model to the second node. In one embodiment, the method further comprises retraining the first ML model based on the information that indicates the one or more ML model uncertainty requirements received from the second node.

In one embodiment, the ML model uncertainty requirements comprise ML model prediction uncertainty value requirements for the first node to satisfy in regard to the one or more output predictions sent to the second node. In one embodiment, the ML model prediction uncertainty value requirements specify uncertainty information to be provided by the first node. In one embodiment, the ML model prediction uncertainty value requirements comprise conditions under which the first node should not provide ML model output predictions.

In one embodiment, the method further comprises sending, to the second node, capability information that indicates one or more capabilities of the first node related to computing and/or reporting uncertainty of the one or more output predictions of the first ML model. In one embodiment, the capability information comprises information that indicates that the first node supports computing and/or reporting of per-sample prediction uncertainty during inference, information that indicates that the first node supports computing and/or reporting of statistical uncertainty from training, information that indicates that the first node supports computing and/or reporting of epistemic uncertainty in inference, or any combination thereof.

In one embodiment, the first node and/or the second node is a network node of a cellular communications system.

In one embodiment, the first node is a User Equipment (UE), and the second node is a network node in a Radio Access Network (RAN), wherein the one or more output predictions comprise one or more signal quality predictions for use in a handover decision.

In one embodiment, the first node is a network node in a RAN, and the one or more output predictions comprise one or more load predictions and/or one or more energy-saving predictions for the network node.

Corresponding embodiments of a first node are also disclosed. In one embodiment, a first node is adapted to send, to a second node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions.

In one embodiment, a first node comprises processing circuitry configured to cause the first node to send, to a second node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions.

Embodiments of a method performed by a second node are also disclosed. In one embodiment, a method performed by a second node comprises receiving, from a first node, one or more output predictions from a first ML model together with uncertainty information about the one or more output predictions, and using the one or more output predictions and uncertainty information in combination with one or more other inputs to perform one or more actions related to operation of a radio network. In one embodiment, the uncertainty information comprises information that indicates one or more types of uncertainty the one or more output predictions are subject to. In one embodiment, the uncertainty information further comprises information about the one or more types of uncertainty the one or more output predictions are subject to.

In one embodiment, the method further comprises sending, to the first node, information that indicates one or more ML model uncertainty requirements.

In one embodiment, the method further comprises receiving, from the first node, capability information that indicates one or more capabilities of the first node related to computing and/or reporting uncertainty of the one or more output predictions of the first ML model. In one embodiment, the capability information comprises information that indicates that the first node supports computing and/or reporting per-sample prediction uncertainty during inference, information that indicates that the first node supports computing and/or reporting statistical uncertainty from training, information that indicates that the first node supports computing and/or reporting epistemic uncertainty in inference, or any combination thereof.

Corresponding embodiments of a second node are also disclosed. In one embodiment, a second node is adapted to receive, from a first node, one or more output predictions from a first ML model together with uncertainty information about the one or more output predictions and use the one or more output predictions and uncertainty information in combination with one or more other inputs to perform one or more actions related to operation of a radio network.

In one embodiment, a second node comprises processing circuitry configured to cause the second node to receive, from a first node, one or more output predictions from a first ML model together with uncertainty information about the one or more output predictions and use the one or more output predictions and uncertainty information in combination with one or more other inputs to perform one or more actions related to operation of a radio network.

Brief of the

The accompanying drawing figures incorporated in and forming a part of this specification illustrate several aspects of the disclosure, and together with the description serve to explain the principles of the disclosure. Figure 1 illustrates an example of chained Machine Learning (ML) models;

Figure 2 illustrates examples of aleatoric and epistemic uncertainties in predictions made via ML or Artificial Intelligence (Al);

Figure 3 illustrates an example of uncertainty estimation using different ML models;

Figure 4 is a flowchart of methods in accordance with embodiments of the present disclosure;

Figure 5 depicts a method in accordance with particular embodiments of the present disclosure;

Figure 6 a method in accordance with some other particular embodiments of the present disclosure;

Figure 7 shows a schematic of a portion of a network;

Figure 8 illustrates uncertainty reported using gaussian mixture with two components namely, a first component with mean =-100, sigma = 1 , component weight = 113, and a second component with mean =-90, sigma = 1, component weight =2/3;

Figure 9 shows predicted User Equipment (UE) bitrates at different expected load values in target cell, where 0.5 means that 50% of the time-frequency resources are expected to be allocated to other UEs;

Figure 10 illustrates an example of an uncertainty interval in accordance with an embodiment of the present disclosure;

Figure 11 shows an example of a communication system 1100 in accordance with some embodiments of the present disclosure;

Figure 12 shows a UE in accordance with some embodiments of the present disclosure;

Figure 13 shows a network node in accordance with some embodiments of the present disclosure;

Figure 14 is a block diagram of a host, which may be an embodiment of the host 1116 of Figure 11, in accordance with various aspects described herein;

Figure 15 is a block diagram illustrating a virtualization environment in which functions implemented by some embodiments may be virtualized; and

Figure 16 shows a communication diagram of a host communicating via a network node with a UE over a partially wireless connection in accordance with some embodiments.

Detailed Description

The embodiments set forth below represent information to enable those skilled in the art to practice the embodiments and illustrate the best mode of practicing the embodiments. Upon reading the following description in light of the accompanying drawing figures, those skilled in the art will understand the concepts of the disclosure and will recognize applications of these concepts not particularly addressed herein. It should be understood that these concepts and applications fall within the scope of the disclosure.

Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

There currently exist certain challenge(s). When signaling predictions from a Machine Learning (ML) model, it is typically important that the receiving node of such prediction also receives an associated uncertainty. The method on how the uncertainty is estimated at the first node may not be known by the receiving node; hence, the received uncertainty report might be not adequate for the receiving node. With reference to Figure 3, if a first node estimates a given uncertainty level for the outputs signaled to a second node, the first node may report to the second node the same level of uncertainty for outputs derived by samples within the range of 2<x<4 as for all other outputs. The second node and the model hosted by it, which uses the output of the first node as its inputs, will hence not consider the fact that the output provided by the first node derived by samples within the range of 2<x<4 is the result of a training process that had no samples within the range of 2<x<4. As a consequence, the second node may act erroneously and wrongly consider the predictions received from the first node for the intervals that lack training data at the first node. As an example of such an erroneous action, if a second node receives a load prediction from a first node, where the first node derives such prediction from a data sample in a certain interval where it does not have any training data, the prediction produced may be subject to unknown uncertainty. If a second node receiving such prediction is not aware of the unknown uncertainty, it can derive an inaccurate uncertainty estimation, which may lead to potentially inefficient load balancing decisions at the second node. Accordingly, it is desirable to expose the nature of the uncertainty affecting a given output prediction signaled from a first node to a second node.

Selecting a model that supports taking epistemic uncertainty into account may increase the complexity of the overall system, leading to more complex models. A potential consequence of the use of more complex models is the use of large amounts of computational power to provide accurate uncertainty estimates for the receiving/second node. Accordingly, it is desirable to select a suitable model providing output predictions that do not generate excessive computational load for the system.

The first (transmitting) node is typically not aware of the required uncertainty accuracy level needed by the second (receiving) node. A second node may not utilize the uncertainties from a first node; where this is the case, if the first node spends resources in calculating the uncertainty associated with a given output signaled to the second node, the first node might use an overly complex model and such effort may be spent in vain. Also, there is typically no indication from the second node to the first node on the acceptable level of uncertainty regarding a model output. The lack of indication can cause of lot of unnecessary overhead in terms of signaling predictions that anyway cannot be used by the second node. Where an output of a model is provided as input to a model/algorithm in the second node, it may not be possible to tailor the output of the model to the needs of the model/algorithm in the second node. A consequence of this lack of tailoring may be poor performance of the model/algorithm in the second node. Further, a node receiving a prediction (e.g., the second node in the present example) may not want/need to receive predictions, depending on the type and extent of uncertainty of the predicted values or depending on the type of parameters to which predictions - affected by a certain type and/or extent of uncertainty - refer.

Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. Embodiments provide methods in second (receiving) network nodes to control and ensure that ML-models in first (transmitting) network nodes may train models to accurately describe an uncertainty associated to the predictions, and may select what type of uncertainty format the first network node should use. The predictions from the model in a first network node may then be used in a second network node (actor node). A flowchart of methods in accordance with embodiments of the disclosure is shown in Figure 4.

Figure 4 shows scenarios in which a ML model (at a training/inference node, node 1) that performs inference from a given set of inputs provides its outputs to a second node/model (the actor node, node 2), which in turns performs inference on its own set of inputs. The inference can be both accomplished by a data driven ML-model (option 1) or any algorithm built using domain competence (option 2). The first node may also be referenced herein as first node 400, and the second node may also be referenced herein as second node 402. Additional detail on embodiments of the disclosure, with reference to the steps in Figure 4 are provided below. Node 1 (also referred to as a further network node) and node 2 (also referred to as a network node) may either or both be a Radio Access Network (RAN) node, an Operations and Maintenance (OAM) node, a Core Network (ON) node, a Service Management and Orchestration (SMO) node, a Network Management System (NMS), a Non-Real Time RAN Intelligent Controller (Non-RT RIO), a Real-Time RAN Intelligent Controller (RT-RIC), a next generation Node B (gNB), an evolved Node B (eNB), Evolved Universal Terrestrial Radio Access Network (E-UTRAN) and New Radio (NR) Dual Connectivity (DCO (EN-DC) gNB (en-gNB), next generation eNB (ng-eNB), gNB Central Unit (gNB-CU), a gNB-CU Control Plane part (gNB-CU-CP), a gNB-CU User Plane part (gNB-CU-UP), an eNB Central Unit (eNB-CU), an eNB-CU Control Plane part (eNB-CU-CP), an eNB-CU User Plane part (eNB-CU-UP), an Integrated Access and Backhaul (IAB) node, an lAB-donor Distributed Unit (DU), lAB-donor Central Unit (CU), an IAB node DU (IAB-DU), an IAB node Mobile Termination (IAB-MT), Open Central Unit (O-CU), O-CU CP part (O-CU-CP), O-CU UP part (O-CU- UP), Open DU (O-DU), Open Radio Unit (O-RU), Open eNB (O-eNB), or a User Equipment (UE).

The steps of the procedure of Figure 4 are as follows. In step 100, a first model (which may be hosted by a further network node which may also be referred to herein as a "training/inference node”, "first node”, or "node 1”) may indicate (to a network node, which may also be referred to herein as an "actor node”, "second node”, or "node 2”), together with its outputs, information concerning how the uncertainty of the outputs and the type of uncertainty is derived, that is, information on how ML model uncertainties are calculated. In other words, the first node provides, to the second node, information concerning how the uncertainty of the outputs of the first model is derived and the type of uncertainty of the outputs of the first model that are derived. The information may consist of capabilities in uncertainty metrics, which may include one or more of the following indications:

• Support of (computing and/or reporting of) per-sample prediction uncertainty during inference (can also be called inference output accuracy), using methods such as Gaussian processes, or any method where the prediction uncertainty for two unique ML-model inputs are different o Given support for per-sample prediction uncertainty, support for such uncertainty to be represented as for example:

■ mean/variance, min/max, confidence interval, and/or prediction interval for each ML model input sample

• Thus, in other words, the uncertainty information may include mean/variance, min/max, confidential interval, and/or prediction interval for each ML model input sample. This information is associated to the respective output prediction from the ML model.

• The uncertainty is associated to the outputs, but any two (sufficiently) distinct inputs can result in the same output value but with different uncertainty. This is in contrast to, e.g., the accuracy of a model, which is an aggregated metric, and it is thus the same for all inputs.

■ point prediction with any associated uncertainty value/metric

• E.g., point prediction X ± Y, where the ±Y corresponds to the uncertainty.

■ probabilistic prediction (e.g., a Probability Mass Function, PMF, and/or Probability Density Function, PDF)

• A PDF can be obtained using for example ensemble-based methods.

• In other words, the uncertainty information may be included in a probabilistic prediction (e.g., PMF and/or PDF). Note that, in some cases, the output of the ML model may already be a probability function. In that case, there may not be a point prediction, or, e.g., the mean is considered as a point prediction. Either way, the uncertainty information is comprised in the probability function.

• In a regression problem, the PDF could be as simple as specifying "Gaussian PDF with mean X and variance Y”, a more complex mixture PDF, or a list of tuples (x,y) defining a curve. In the case of a classification problem, the PMF is directly the output of the model using, e.g., a softmax function. For example, with 3 classes, the output could be: 15%, 80%, 5%.

■ prediction set or region/interval (e.g., conformal or set-valued prediction)

• Note that a prediction set can be used, e.g., in the case of classification. For example, if there are 10 classes, we could do a point prediction and say it is probably class 1 . But perhaps the probability for class 1 is only 50%. We could also predict a class set that has a 97.5% probability of including the correct class, saying it's most likely {class 1, class 5, class 10}, also inherently comprising uncertainty info. In the case of regression or continuous value prediction, we may obtain a prediction interval or region, which also works for multi-dimensional values, i.e., multi-dimensional outputs. o For each of the above, the uncertainty information depends on the specific input and is generally different for each distinct input.

• Support for (computing and/or reporting of) statistical uncertainty from training o Indicating homoscedastic uncertainty with associated value (e.g., mean/variance). Using any method where the prediction uncertainty, for two unique ML-model inputs, is equal

■ This can be provided together with the input or input range around which such statistical uncertainty is produced

■ This can be provided together with statistics on how many data samples that was included when estimating the statistical uncertainty

■ This can be provided together with description of what ML-model is used to produce the predictions (e.g., neural network, random forest etc.) o Indicating heteroscedastic uncertainty with associated value per output range value. Using any ML-method where the prediction uncertainty for two ML-model outputs can be different.

• Support for including (computing and/or reporting of) epistemic uncertainty in inference (indicate when there is no or not enough training data). In these cases, the output may be subject to an uncertainty that is not validated by the training process. The uncertainty may be calculated as an extrapolation with respect to output values for which an uncertainty has been validated by a training process, and for that, such uncertainty may be subject to errors o The output from the model may be associated to an indication that the model has not been sufficiently trained to derive a reliable measure of the output uncertainty. This information may be signaled on a per output value basis and might be associated to one input or a range of inputs for which epistemic uncertainty affects the related output o i.e., model 1 is capable of answering "I don't know”, i.e. the uncertainty is higher than a certain threshold where model 1 can indicate in a flag that it cannot provide any prediction. The threshold can be received from model 2 as described in the following. o For example, based on anomaly/outlier detection techniques using, e.g., isolation forests, autoencoders, one-class SVMs, etc.

In addition to the capabilities information, the first node may report a complexity metric associated with each capability support, for example reporting a per-sample uncertainty can require a more complex model in comparison to only report a statistical/aggregated uncertainty metric from the training phase. The complexity may be a discrete value between [1 , N], where 1 means a simple model, and N means a complex method associated to high memory and energy consumption.

Step 100 is not necessarily performed, that is, the first node may not signal to the second node its capabilities in terms of how the uncertainty is calculated. Where step 100 is omitted, the second node may still signal to the first node its requirements in terms of uncertainty affecting the first node outputs signaled to the second node (see step 110 below).

In step 105, the actor node may determine an uncertainty metric format (UMF). In step 110, the actor host (node 2, network node) may indicate (for example, in a signaling message), model uncertainty requirements to node 1 (further network node), for example by indicating the UMF to the node 1 hosting ML model 1 . Such indication allows node 1 to signal ML outputs with a certain uncertainty format (based on its supported uncertainty metrics, which may be provided in step 100). The message in Step 110 may indicate, for example, that model l/node 1 should provide per-sample uncertainty for each prediction. The node 2 can select such uncertainty metric based on its capabilities in using an uncertainty when, for example, training the second ML-model. In some embodiments, particularly where information has been received from the further network node (host 1) in step 100, ML-model host 2 may decide not to receive predictions from host 1 due to lack of support at ML model 1 for specific UMFs or when a specific uncertainty format is associated with a too high complexity.

In some embodiments, if step 100 does not take place and/or if the predictions received by node 2 do not comply with node 2's requirements on prediction uncertainty, node 2 may decide to discard the predictions and to not use them, or to assign a limited weight to the predictions in the decisions that would be impacted by such predictions. Predictions that do comply with the requirements may be assigned higher weights.

In some embodiments, regardless of whether or not Step 100 has been performed, Node 2 may request to Node 1 to report or to not report predictions only if/when such predictions are affected by a certain type of uncertainty or to report or to not report predictions depending on an overall acceptable level of uncertainty or depending on an acceptable level per uncertainty type. As an example, Node 2 can request Node 1 to not report predictions affected by epistemic uncertainty.

In some embodiments, regardless of whether or not Step 100 has been performed, Node 2 may request to Node 1 to pause/resume/stop/start reporting of predictions at an acceptable level of uncertainty or to pause/resume/stop/start reporting of predictions affected by a certain uncertainty type. One reason for that could be that Node 2 or Node 1 or both is/are operating in special conditions, and producing/transmitting or receiving/using predictions affected by a certain level of uncertainty and/or predictions affected by a certain type of uncertainty would be of no or little use or even detrimental. An example of special condition may be that Node 2 would request Node 1 to pause sending of certain predictions when at overload or close to overload and Node 2 would request to resume when the load diminishes. A further example of special condition may be related to a desirable strategy for load balancing between Node 2 and Node 1, wherein transferring user would be allowed/desirable upon fulfillment of certain criteria (e.g., load metric "X” has value higher than "Threshold”). If Node 2 uses measurements and predictions from Node 1 to evaluate load balancing actions towards Node 1 based (among others) on metric "X”, then Node 2 may request Node 1 to refrain from sending predictions for load metric "X” when its current load according to the same metric is close to "Threshold” and the metric "X” is affected by a not-acceptable level of uncertainty or is affected by epistemic uncertainty.

In some embodiments, regardless of whether or not Step 100 has been performed, Node 2 can inform Node 1 of a new or updated acceptable level of uncertainty or an updated type of acceptable uncertainties. This can happen, e.g., as part of a configuration update sent from Node 2 to Node 1 .

In another embodiment of the method, the UMF information further includes acceptable levels of uncertainty of the model output 1 . The second node may determine the said uncertainty level based on various information, e.g., the importance of model Ts output for the processes carried out by model 2 or the sensitivity of model 2’s output to said model 1 's output, and report the acceptable uncertainty level to the first node. The second node may hence only accept model Ts output when a certain uncertainty level is not exceeded. This can significantly reduce the signaling overhead, particularly useful in case of over-the-air signaling between UE and gNB. In one example, the UE sends traffic forecasts to the gNB and such forecast may be more certain at times when the user streams a video and less certain at times when the user browses the web, due to the more continuous traffic pattern of video streaming applications. In the latter example, the traffic forecast may be reported only when the UE has higher certainty of the forecast. In another example, ML model 1 hosted in RAN node 1 will only report its load prediction output if the output is subject to aleatoric uncertainty lower than a given threshold.

In some embodiments, if desired, Node 1 hosting model 1 may signal its outputs to Node 2 hosting model 2 together with the additional information on uncertainty, as shown in step 120 of Figure 4. Such uncertainty information may have been derived and refined by means of the signaling in Step 100 and/or Step 110, or the uncertainty information might be provided by node 1 without any previous exchange of information on uncertainty capabilities (from node 1 to node 2) and/or uncertainty requirements (from node 2 to node 1). Where node 1 transmits information to node 2 (or causes such information to be transmitted), the transmission may be a discrete transmission or a continuous stream of data.

The uncertainty information signaled by node 1 may include one or more of:

• the type or types of uncertainty the prediction is subject to (for example aleatoric, epistemic);

• further details on the type of uncertainty, for example, for aleatoric uncertainty, whether this is homoscedastic or heteroscedastic; and the level of uncertainty.

The uncertainty information may also specify one or more of:

• Whether the uncertainty provided is per-sample prediction uncertainty during inference (can also be called inference output accuracy), using methods such as Gaussian processes, or any method where the prediction uncertainty for two unique ML-model inputs are different. o Given support for per-sample prediction uncertainty, support for such uncertainty may be represented as for example:

■ point prediction with any associated uncertainty value/metric

■ probabilistic prediction (e.g., PMF or PDF)

• A PDF can be obtained using for example ensemble-based methods

• Whether the uncertainty provided is statistical uncertainty from training o Indicate homoscedastic uncertainty with associated value (e.g., mean/variance). Using any method where the prediction uncertainty, for two unique ML-model inputs, is equal:

■ This can be provided together with the input or input range around which such uncertainty is produced by an output

■ This can be provided together with description of what ML-model is used to produce the predictions (e.g., neural network, random forest etc.) o Indicate heteroscedastic uncertainty with associated value per output range value. Using any ML- method where the prediction uncertainty for two ML-model outputs can be different: ■ This can be provided together with the input or input range around which such uncertainty is produced by an output

• Whether the uncertainty provided includes epistemic uncertainty in inference (indicate when there is no or not enough training data). In these cases, the output may be subject to an uncertainty that is not validated by the training process. The uncertainty may be calculated as an extrapolation with respect to output values for which an uncertainty has been validated by a training process, and for that, such uncertainty may be subject to errors. o The output from the model is associated to an indication that the model has not been sufficiently trained to derive a reliable measure of the output uncertainty. This information may be signaled on a per output value basis and might be associated to one input or a range of inputs for which epistemic uncertainty affects the related output o i.e., the model 1 is capable of answering "I don't know”, i.e. the uncertainty is higher than a certain threshold where model 1 can indicate in a flag that it cannot provide any prediction. The threshold can be received from model 2 as described in subsequent sections o For example, based on anomaly/outlier detection techniques using, e.g., isolation forests, autoencoders, one-class SVMs, etc.

Any information received in Step 110 may require the first node to retrain its model or to select a different (pre-trained) model to use a ML model configuration that satisfies the ML model uncertainty requirements, as shown in step 115. For example, the first node may retrain or switch models to use ML-models that supports estimating the epistemic uncertainty as requested/required for the second node (node 2 can request to not receive uncertain values, either if the epistemic uncertainty is high and/or if the predicted uncertainty is high).

In some embodiments the first node may use the acceptable levels of uncertainty that may be included in the UMF to simplify ML model training/selection and reduce computational complexity by training or selecting the simplest (least complex) model that fulfills the given uncertainty requirement, and/or to know when not to signal the model output 1 to the second node or when to respond with "I don't know” or when to mark the model output 1 as unreliable.

In steps 125 and 130, the network node may perform an inference process, potentially using output produced by the further network node where such an output has been received and is satisfactory. Via such input and via the uncertainty information received from ML model 1, node 2 can perform a number of actions such as:

• Assigning a specific weight to the output from ML model 1 , such weight depending on the uncertainty information associated it.

• Evaluating the uncertainty of its own output and in turn forward such information to nodes that subscribe to node 2's outputs, if needed. • Deducing whether the output from ML model 1 shall be considered for the purpose of deriving information/actions at node 2, or whether the uncertainty associated to the output from ML model 1 is such that it should be discarded

• Deducing whether any possible inference (e.g., predictions on actions/information) can be run by means of the output from ML model 1, or if inferences of such type shall not be produced (e.g., due to too high uncertainty levels).

In some embodiments, if node 2 has an ML model and uses the outputs from ML model 1 in its own training process, it may use the uncertainty information provided to optimize the training, e.g., by requesting more data or modifying its training parameters. If the uncertainty in its own predicted output is too high or unacceptable, ML model 2 may decide to stop the training process and move to Step 140, in case it can force ML model 1 into reducing the uncertainty of its outputs.

In some embodiments, the second node can use the static statistical uncertainty report (i.e., from training) provided by the first node to (re-)train ML model 2 using techniques that can incorporate the given input uncertainty so that ML model 2 becomes (more) robust against the uncertainty regarding the output of ML model 1 . As an example of this, the second node may (re-)train ML model 2 with jitter in the input data based on the received input uncertainty, meaning that it adds a certain (small) noise term to each training sample from the training dataset.

In another option, the actor node may use the received prediction and the received uncertainty information with other inputs for a non-ML based algorithm in a radio network operation, as shown in step 135 of Figure 4.

In step 140, ML model 2 may notify ML model 1 of the performance of its generated output, that is, the network node may provide model performance feedback to the further network node. ML model 2 may signal to ML model 1 the uncertainty achieved for its outputs, given the inputs provided by ML model 1 . In some embodiments of this method, ML model 2 may also signal the uncertainty that might have been produced if the inputs (model 1 outputs) had uncertainty within a certain range or above/below a certain threshold. ML model 1 may use this information to improve its outputs, which are signaled to ML model 2. For example, ML model 1 may decide to retrain its model with a set of inputs (independent variables) that, e.g., resolves aleatoric uncertainty in certain areas, e.g. by considering more inputs (independent variables) accepting that the model and the training process becomes more complex. In another example, ML model 1 may decide to re-train its model with another, i.e. larger or newer, dataset that, e.g., resolves epistemic uncertainty in certain areas. ML model 1 may return to Step 120, and the process continues from that point again.

Embodiments thereby provide methods network nodes to control and ensure that an ML-model in a further network node properly trains the model to accurately describe an uncertainty associated to the predictions.

Certain embodiments may provide one or more of the following technical advantage(s). Embodiments may support controlling the uncertainty estimation in node 1 from node 2. Through this control, the robustness/consistency of the models/algorithms that use the prediction with associated uncertainty from model 1 may be improved, for example to take load-balancing or energy saving decisions using the said prediction. The increased robustness may result from ensuring that ML-model 1 supports epistemic uncertainty estimation and/or heteroscedastic uncertainty estimation. Embodiments may enable more efficient operations in node 1 (improved energy efficiency, less memory /process! ng resources spent on ML-model 1), since node 2 can tailor the desired uncertainty metric format based on one or more of: its capabilities/processing constraints in using a prediction from node 1; use case requirements on explainability/accurate uncertainties; and/or computational complexity indicated by node 1, (to only use more computational heavy methods that can consider epistemic uncertainty if needed). Embodiments may also provide reduced signaling overhead by avoiding transmission of predictions (model 1 outputs) with unacceptable levels of uncertainty.

Figure 5 depicts a method in accordance with particular embodiments. The method 5 may be performed by a further network node (e.g., the UE 1112 or UE 1200 as described later with reference to Figures 11 and 12 respectively, or the network node 1110 or network node 1300 as described later with reference to Figures 11 and 13 respectively). The method begins at step 502 with receiving, from a network node, ML model uncertainty requirements. The method continues at step 504 with, in response to the received ML model uncertainty requirements, determining whether or not to initiate transmission (by transmitting by the further network node or by causing a further component to transmit), to the network node, of ML model information. The method then continues at step 506 with, where it is determined to initiate transmission of the ML model information, initiating transmission of the ML model information to the network node.

Figure 6 depicts a method in accordance with particular embodiments. The method 6 may be performed by a network node (e.g., the UE 1112 or UE 1200 as described later with reference to Figures 11 and 12 respectively, or the network node 1110 or network node 1300 as described later with reference to Figures 11 and 13 respectively). The method begins at step 602 with initiating transmission, to a further network node, of ML model uncertainty requirements. The method continues at step 604 with performing, at the network node, ML modelling (potentially using ML model information received from the further network node).

UE Forecasted Signal Quality Application for Mobility

In an example, the network is interested in learning the best mobility decision (radio network operation in signaling chart of Figure 4) for a device. The device first may indicate its supported capabilities in providing an uncertainty metric in addition to its predicted coverage on a target cell, optionally including the computational complexity associated to providing the uncertainty (see step 100 of Figure 4). The UE may predict the target cell coverage operating on a secondary carrier, for example using the techniques described in "Predicting strongest cell on secondary carrier using primary carrier data” by Ryden, H. et al, available at https://ieeexplore.ieee.org/abstract/document/8369000 as of 4 August 2021 .

In this regard, Figure 7 shows a schematic of a portion of a network 700. As illustrated in Figure 7, the network 700 includes a serving node 702 of a UE 704, a target node 706 for the UE 704, and a neighbor node 708.

If it is assumed that the network wants to perform a blind inter-frequency handover, meaning that the UE does not need to do any measuring before being handed over (as an example of option 2 from Figure 4), the network then needs accurate predictions, and can require the UE to support aleatoric and/or epistemic uncertainty estimation, and support reporting the predicted signal coverage uncertainty as a probability density function (PDF). Reporting it as a PDF will allow the network to understand the probable signal qualities for the device on another cell. In case the network does not intend to perform a blind handover, it can request the UE to relax its uncertainty constraints, enabling the UE to select a less complex model. The signal quality prediction can, as one example, be reported using the parameters describing the mixed gaussian components (mean, variation, and component weight for each of the components), or represented using a histogram as illustrated in Figure 8, one p(x) for each bin x=[-120,-60] .

Figure 8 illustrates uncertainty reported using gaussian mixture with two components. Component 1 : mean =-100, sigma = 1, component weight = 1/3. Component 2: mean =-90, sigma = 1, component weight =2/3. Representing an uncertainty metric as a single value might not be adequate for all scenarios, for example in the figure, if only supporting one value, one would take the overall standard deviation of the PDF. This type of one-value is not adequate for all problems, some problems are by nature not suitable for reporting a one-value uncertainty. Node 2 can request the node 1 to report an uncertainty in a per-sample PDF.

If the UE signal quality is used as input to a second ML model (also called chained ML model) (see option 1 of Figure 4). Where it is assumed that the second node has trained a second model, where the load and the predicted signal quality in a target cell are used as inputs to predict the bitrate in the said target cell, the second node may, for example, sample from the UE-reported PDF and combine with a load forecast in the target cell to estimate a distribution of potential bitrates. At a fixed load level at the target node, as illustrated in Figure 9, the bitrate prediction will have similar characteristic as the PDF reported from the UE. The bitrate prediction can then be used to take a handover decision using any algorithm if bitrate is above a certain threshold with a probability. This will hence propagate into an accurate uncertainty measure by the second node.

Figure 9 shows predicted UE bitrates at different expected load values in target cell, where 0.5 means that 50% of the time-frequency resources are expected to be allocated to other UEs.

Network Forecasted Load for Traffic Steering

In a further example, a RAN node (node 1) has an ML model that predicts its own future load (e.g., its load in the next 10-minute interval). A neighboring RAN node (node 2) may subscribe to node Ts prediction and require a certain level of uncertainty, or a certain level of uncertainty for a given type of uncertainty in those predicted load values, or require to receive/not receive predicted load values depending on a level of uncertainty, a type of uncertainty (e.g., not receive prediction if epistemic uncertainty is present), or a combination thereof.

With reference to option 2 of Figure 4, node 2 may decide to hand over a certain number of UEs to node 1 if the predicted load (including the uncertainty) is below a predefined threshold. Moreover, the number of UEs handed over to node 1 may depend on the uncertainty of the predicted load by node 1 . An example of the uncertainty interval is shown in Figure 10.

In Figure 10, uncertainty is reported as a min/max interval. In this example, the node load should stay below 80% and the predicted value is 70%. In scenario 1, the uncertainty is large enough to make this prediction unreliable; node 2 may ignore/reject this prediction. In scenario 2, the uncertainty interval is smaller and allows node 2 to plan the handover of UEs to node 1 . It is not unusual that the uncertainty interval is asymmetric; this may happen if it is more likely that the UEs maintain their traffic pattern or disconnect rather than increase their utilization of resources.

With reference to option 1 of Figure 4, node 2 may host an ML model that predicts node Ts future load after node 2 has handed over a certain number of UEs. The inputs to this ML model could be node Ts predicted load (without any knowledge of node 2's future decision) and the traffic pattern of the potential UEs to be transferred. It is also possible that the traffic pattern of the UEs was predicted by the UEs themselves. Therefore, node 2 may require the reporting of uncertainty not only to node 1, but also to the UEs. It is up to node 2 to combine all the predictions, and their uncertainties, to produce a successful load balancing strategy.

Example Uncertainty Metrics

There are a variety of techniques for quantifying the uncertainty associated to an output (i.e. , a prediction) of an ML model. In many cases, depending on the ML model, the uncertainty of an output is estimated as part of ML inference or can be derived from the output itself. The output uncertainty may be inherent in the output or expressed separately. Two examples follow.

In a first example, classification with uncertainty estimation may be used. A ML model is used for multi-class classification. Many models perform classification by predicting the probability that an input belongs to a certain class, e.g., using a One-vs-Rest (OvR) scheme. The input is assigned a certain class X with a certain probability P_x. Thus, the probability that the input belongs to any other class is given by the probability 1 - P_x. The output (i.e., prediction) can be given as point estimate (i.e., class X) with the uncertainty value of 1 - P_x.

In a second example, regression with uncertainty estimation may be used. A Bayesian neural network may be used for regression. In this model, the weights (and biases) are considered probability distributions, instead of single values (point estimates). In one approach, the weight (and bias) distributions are sampled to produce a point prediction for a given input. By comparing the predictions of multiple sampled model parametrizations for a given input, such approach enables a quantification of epistemic uncertainty of the prediction. The output may be given as point estimate, e.g., the average of the predictions, with an associated uncertainty value, e.g., the variance of the predictions. Alternatively, the output may be given as histogram or empirical PDF/PMF, which inherently comprises a measure of uncertainty.

Example Uncertainty Information

Uncertainty information may be provided by node 1 to node 2 in any suitable way. An example is as follows.

In the present example, it is shown how the uncertainty information may be signaled from node 1 to node 2, together with the inference data (e.g., predictions) signaled from nodel to node 2. The information comprising the uncertainty information may be structured as shown in Table 1 below, which shows an example Information Element (IE). The IE contains information about the type and level of uncertainty associated with a given set of data.

Table 1 : Example IE

The information in Table 1 may be included in a message from node 1 to node2 together with the set of data to which the uncertainty information is associated. Similar information may also be included in a message from Node 2 (the node due to receive predictions) to Node 1 (the node producing the prediction) to inform Node 1 about type and corresponding level of uncertainty. Node 2 may also request Node 1 to limit/filter the type of predictions to be sent to Node 2 based on uncertainty types and levels. In this case, the information can be signaled during a registration phase, prior to the sending of predictions from Node 1 to Node 2.

A potential implementation of an IE such as that discussed above, related to the exchange of load metrics, is as follows:

• Node 2 initiates a registration request, e.g. as comprised in a XnAP RESOURCE STATUS REQUEST message, and includes in such message one or more of: o signaling information indicating to Node 1 to send the type of Uncertainty together with a prediction; o signaling information indicating to Node 1 to send predictions for load metrics comprised in Radio Resource Status only if Uncertainty type-' Aleatoric” has an Uncertainty level = "10”; and/or o signaling information indicating to Node 1 to not send predictions for load metrics comprised in Radio Resource Status if Uncertainty type = "Epistemic”.

• Node 1 receives the registration request and sends to Node 2 a XnAP RESOURCE STATUS RESPONSE message, indicating that the request is acknowledged.

• Node 1 sends load metrics using an XnAP RESOURCE STATUS UPDATE message, and, with reference to the registration request, may perform one or more of the following: o include predictions of load metrics related to Radio Resource Status when the assessed uncertainty level for uncertainty type "Aleatoric” is lower than or equal to 10; o not include predictions of load metrics related to Radio Resource Status when the assessed uncertainty level for uncertainty type "Aleatoric” is higher than 10; o not include predictions for load metrics comprised in Radio Resource Status if Uncertainty type is "Epistemic”; and/or o Node 1 may inform Node 2 that reason for not sending predictions of load metrics related to Radio Resource Status is due to unsatisfactory level of "Aleatoric” uncertainty or due to the presence of uncertainty of type "epistemic”.

If Node 2 receives many empty samples of predictions for Radio Resource Status, it may perform different actions, e.g.: 1) ignore the situation; 2) stop requesting predictions to Node 1; 3) request Node 1 to improve the uncertainty

Additional Description

Figure 11 shows an example of a communication system 1100 in accordance with some embodiments.

In the example, the communication system 1100 includes a telecommunication network 1102 that includes an access network 1104, such as a radio access network (RAN), and a core network 1106, which includes one or more core network nodes 1108. The access network 1104 includes one or more access network nodes, such as network nodes 1110a and 1110b (one or more of which may be generally referred to as network nodes 1110), or any other similar 3^rd Generation Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 1110 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 1112a, 1112b, 1112c, and 1112d (one or more of which may be generally referred to as UEs 1112) to the core network 1106 over one or more wireless connections. The network node (second network node, actor node) and further network node (first network node, training/inference node) may be any of the nodes shown in Figure 11 .

Example wireless communications over a wireless connection include transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and/or any other components or systems that may facilitate or participate in the communication of data and/or signals whether via wired or wireless connections. The communication system 1100 may include and/or interface with any type of communication, telecommunication, data, cellular, radio network, and/or other similar type of system.

The UEs 1112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and/or operable to communicate wirelessly with the network nodes 1110 and other communication devices. Similarly, the network nodes 1110 are arranged, capable, configured, and/or operable to communicate directly or indirectly with the UEs 1112 and/or with other network nodes or equipment in the telecommunication network 1102 to enable and/or provide network access, such as wireless network access, and/or to perform other functions, such as administration in the telecommunication network 1102.

In the depicted example, the core network 1106 connects the network nodes 1110 to one or more hosts, such as host 1116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1106 includes one more core network nodes (e.g., core network node 1108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and/or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1108. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and/or a User Plane Function (UPF).

The host 1116 may be under the ownership or control of a service provider other than an operator or provider of the access network 1104 and/or the telecommunication network 1102, and may be operated by the service provider or on behalf of the service provider. The host 1116 may host a variety of applications to provide one or more services. Examples of such applications include the provision of live and/or pre-recorded audio/video content, data collection services, for example, retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

As a whole, the communication system 1100 of Figure 11 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and/or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and/or any low- power wide-area network (LPWAN) standards such as LoRa and Sigfox.

In some examples, the telecommunication network 1102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 1102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 1102. For example, the telecommunications network 1102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and/or Massive Machine Type Communication (mMTC)/Massive loT services to yet further UEs.

In some examples, the UEs 1112 are configured to transmit and/or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

In the example illustrated in Figure 11, the hub 1114 communicates with the access network 1104 to facilitate indirect communication between one or more UEs (e.g., UE 1112c and/or 1112d) and network nodes (e.g., network node 1110b). In some examples, the hub 1114 may be a controller, router, a content source and analytics node, or any of the other communication devices described herein regarding UEs. For example, the hub 1114 may be a broadband router enabling access to the core network 1106 for the UEs. As another example, the hub 1114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1110, or by executable code, script, process, or other instructions in the hub 1114. As another example, the hub 1114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 1114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1114 then provides to the UE either directly, after performing local processing, and/or after adding additional local content. In still another example, the hub 1114 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.

The hub 1114 may have a constant/persistent or intermittent connection to the network node 1110b. The hub 1114 may also allow for a different communication scheme and/or schedule between the hub 1114 and UEs (e.g., UE 1112c and/or 1112d), and between the hub 1114 and the core network 1106. In other examples, the hub 1114 is connected to the core network 1106 and/or one or more UEs via a wired connection. Moreover, the hub 1114 may be configured to connect to an M2M service provider over the access network 1104 and/or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1110 while still connected via the hub 1114 via a wired or wireless connection. In some embodiments, the hub 1114 may be a dedicated hub - that is, a hub whose primary function is to route communications to/from the UEs from/to the network node 1110b. In other embodiments, the hub 1114 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1110b, but which is additionally capable of operating as a communication start and/or end point for certain data channels.

Figure 12 shows a UE 1200 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and/or operable to communicate wirelessly with network nodes and/or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless camera, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded/integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-loT) UE, a machine type communication (MTC) UE, and/or an enhanced MTC (eMTC) UE.

A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to- infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and/or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

The UE 1200 includes processing circuitry 1202 that is operatively coupled via a bus 1204 to an input/output interface 1206, a power source 1208, a memory 1210, a communication interface 1212, and/or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 12. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

The processing circuitry 1202 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 1210. The processing circuitry 1202 may be implemented as one or more hardware- implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 1202 may include multiple central processing units (CPUs). The processing circuitry 1202 may be operable to provide, either alone or in conjunction with other UE 1200 components, such as the memory 1210, to provide UE 1200 functionality. For example, the processing circuitry 1202 may be configured to cause the UE 1202 to perform the methods as described with reference to Figure 5.

In the example, the input/output interface 1206 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and/or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 1200. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

In some embodiments, the power source 1208 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 1208 may further include power circuitry for delivering power from the power source 1208 itself, and/or an external power source, to the various parts of the UE 1200 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 1208. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 1208 to make the power suitable for the respective components of the UE 1200 to which power is supplied.

The memory 1210 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 1210 includes one or more application programs 1214, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 1216. The memory 1210 may store, for use by the UE 1200, any of a variety of various operating systems or combinations of operating systems.

The memory 1210 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and/or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUlCC), integrated UICC (IUICC) or a removable UICC commonly known as ‘SIM card.' The memory 1210 may allow the UE 1200 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 1210, which may be or comprise a device-readable storage medium.

The processing circuitry 1202 may be configured to communicate with an access network or other network using the communication interface 1212. The communication interface 1212 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 1222. The communication interface 1212 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 1218 and/or a receiver 1220 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 1218 and receiver 1220 may be coupled to one or more antennas (e.g., antenna 1222) and may share circuit components, software or firmware, or alternatively be implemented separately.

In some embodiments, communication functions of the communication interface 1212 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and/or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol/internet protocol (TCP/IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 1212, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient). As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or controls a robotic arm performing a medical procedure according to the received input.

A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are devices which are or which are embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door/window sensor, a flood/moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a headmounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and/or software in dependence on the intended application of the loT device in addition to other components as described in relation to the UE 1200 shown in Figure 12.

As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and/or measurements, and transmits the results of such monitoring and/or measurements to another UE and/or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-loT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and/or reporting on its operational status or other functions associated with its operation.

In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone's speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g., by controlling an actuator) to increase or decrease the drone's speed. The first and/or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

Figure 13 shows a network node 1300 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and/or operable to communicate directly or indirectly with a UE and/or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)). Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and/or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multistandard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi- cell/multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and/or Minimization of Drive Tests (MDTs).

The network node 1300 includes processing circuitry 1302, a memory 1304, a communication interface 1306, and a power source 1308, and/or any other component, or any combination thereof. The network node 1300 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1300 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1300 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1304 for different RATs) and some components may be reused (e.g., a same antenna 1310 may be shared by different RATs). The network node 1300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1300, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1300.

The processing circuitry 1302 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and/or encoded logic operable to provide, either alone or in conjunction with other network node 1300 components, such as the memory 1304, to provide network node 1300 functionality. For example, the processing circuitry 1302 may be configured to cause the network node to perform the methods as described with reference to Figure 6.

In some embodiments, the processing circuitry 1302 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1302 includes one or more of radio frequency (RF) transceiver circuitry 1312 and baseband processing circuitry 1314. In some embodiments, the radio frequency (RF) transceiver circuitry 1312 and the baseband processing circuitry 1314 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 1312 and baseband processing circuitry 1314 may be on the same chip or set of chips, boards, or units.

The memory 1304 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or non-volatile, non-transitory device-readable and/or computer-executable memory devices that store information, data, and/or instructions that may be used by the processing circuitry 1302. The memory 1304 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and/or other instructions capable of being executed by the processing circuitry 1302 and utilized by the network node 1300. The memory 1304 may be used to store any calculations made by the processing circuitry 1302 and/or any data received via the communication interface 1306. In some embodiments, the processing circuitry 1302 and memory 1304 is integrated.

The communication interface 1306 is used in wired or wireless communication of signaling and/or data between a network node, access network, and/or UE. As illustrated, the communication interface 1306 comprises port(s)/terminal(s) 1316 to send and receive data, for example to and from a network over a wired connection. The communication interface 1306 also includes radio front-end circuitry 1318 that may be coupled to, or in certain embodiments a part of, the antenna 1310. Radio front-end circuitry 1318 comprises filters 1320 and amplifiers 1322. The radio front-end circuitry 1318 may be connected to an antenna 1310 and processing circuitry 1302. The radio front-end circuitry may be configured to condition signals communicated between antenna 1310 and processing circuitry 1302. The radio front-end circuitry 1318 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 1318 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1320 and/or amplifiers 1322. The radio signal may then be transmitted via the antenna 1310. Similarly, when receiving data, the antenna 1310 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1318. The digital data may be passed to the processing circuitry 1302. In other embodiments, the communication interface may comprise different components and/or different combinations of components.

In certain alternative embodiments, the network node 1300 does not include separate radio front-end circuitry 1318, instead, the processing circuitry 1302 includes radio front-end circuitry and is connected to the antenna 1310. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1312 is part of the communication interface 1306. In still other embodiments, the communication interface 1306 includes one or more ports or terminals 1316, the radio front-end circuitry 1318, and the RF transceiver circuitry 1312, as part of a radio unit (not shown), and the communication interface 1306 communicates with the baseband processing circuitry 1314, which is part of a digital unit (not shown).

The antenna 1310 may include one or more antennas, or antenna arrays, configured to send and/or receive wireless signals. The antenna 1310 may be coupled to the radio front-end circuitry 1318 and may be any type of antenna capable of transmitting and receiving data and/or signals wirelessly. In certain embodiments, the antenna 1310 is separate from the network node 1300 and connectable to the network node 1300 through an interface or port.

The antenna 1310, communication interface 1306, and/or the processing circuitry 1302 may be configured to perform any receiving operations and/or certain obtaining operations described herein as being performed by the network node. Any information, data and/or signals may be received from a UE, another network node and/or any other network equipment. Similarly, the antenna 1310, the communication interface 1306, and/or the processing circuitry 1302 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and/or signals may be transmitted to a UE, another network node and/or any other network equipment.

The power source 1308 provides power to the various components of network node 1300 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1308 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1300 with power for performing the functionality described herein. For example, the network node 1300 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1308. As a further example, the power source 1308 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

Embodiments of the network node 1300 may include additional components beyond those shown in Figure 13 for providing certain aspects of the network node's functionality, including any of the functionality described herein and/or any functionality necessary to support the subject matter described herein. For example, the network node 1300 may include user interface equipment to allow input of information into the network node 1300 and to allow output of information from the network node 1300. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1300.

Figure 14 is a block diagram of a host 1400, which may be an embodiment of the host 1116 of Figure 11, in accordance with various aspects described herein. As used herein, the host 1400 may be or comprise various combinations hardware and/or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1400 may provide one or more services to one or more UEs.

The host 1400 includes processing circuitry 1402 that is operatively coupled via a bus 1404 to an input/output interface 1406, a network interface 1408, a power source 1410, and a memory 1412. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 12 and 13, such that the descriptions thereof are generally applicable to the corresponding components of host 1400.

The memory 1412 may include one or more computer programs including one or more host application programs 1414 and data 1416, which may include user data, e.g., data generated by a UE for the host 1400 or data generated by the host 1400 for a UE. Embodiments of the host 1400 may utilize only a subset or all of the components shown. The host application programs 1414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAG, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1414 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1400 may select and/or indicate a different host for over-the-top services for a UE. The host application programs 1414 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

Figure 15 is a block diagram illustrating a virtualization environment 1500 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1500 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized.

Applications 1502 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and/or benefits of some of the embodiments disclosed herein.

Hardware 1504 includes processing circuitry, memory that stores software and/or instructions executable by hardware processing circuitry, and/or other hardware devices as described herein, such as a network interface, input/output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1506 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1508a and 1508b (one or more of which may be generally referred to as VMs 1508), and/or perform any of the functions, features and/or benefits described in relation with some embodiments described herein. The virtualization layer 1506 may present a virtual operating platform that appears like networking hardware to the VMs 1508.

The VMs 1508 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1506. Different embodiments of the instance of a virtual appliance 1502 may be implemented on one or more of VMs 1508, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

In the context of NFV, a VM 1508 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1508, and that part of hardware 1504 that executes that VM, be it hardware dedicated to that VM and/or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1508 on top of the hardware 1504 and corresponds to the application 1502.

Hardware 1504 may be implemented in a standalone network node with generic or specific components. Hardware 1504 may implement some functions via virtualization. Alternatively, hardware 1504 may be part of a larger cluster of hardware (e.g., such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1510, which, among others, oversees lifecycle management of applications 1502. In some embodiments, hardware 1504 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1512 which may alternatively be used for communication between hardware nodes and radio units.

Figure 16 shows a communication diagram of a host 1602 communicating via a network node 1604 with a UE 1606 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 1112a of Figure 11 and/or UE 1200 of Figure 12), network node (such as network node 1110a of Figure 11 and/or network node 1300 of Figure 13), and host (such as host 1116 of Figure 11 and/or host 1400 of Figure 14) discussed in the preceding paragraphs will now be described with reference to Figure 16.

Like host 1400, embodiments of host 1602 include hardware, such as a communication interface, processing circuitry, and memory. The host 1602 also includes software, which is stored in or accessible by the host 1602 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1606 connecting via an over-the-top (OTT) connection 1650 extending between the UE 1606 and host 1602. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1650.

The network node 1604 includes hardware enabling it to communicate with the host 1602 and UE 1606. The connection 1660 may be direct or pass through a core network (like core network 1106 of Figure 11) and/or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

The UE 1606 includes hardware and software, which is stored in or accessible by UE 1606 and executable by the UE's processing circuitry. The software includes a client application, such as a web browser or operatorspecific "app” that may be operable to provide a service to a human or non-human user via UE 1606 with the support of the host 1602. In the host 1602, an executing host application may communicate with the executing client application via the OTT connection 1650 terminating at the UE 1606 and host 1602. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1650 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1650.

The OTT connection 1650 may extend via a connection 1660 between the host 1602 and the network node 1604 and via a wireless connection 1670 between the network node 1604 and the UE 1606 to provide the connection between the host 1602 and the UE 1606. The connection 1660 and wireless connection 1670, over which the OTT connection 1650 may be provided, have been drawn abstractly to illustrate the communication between the host 1602 and the UE 1606 via the network node 1604, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

As an example of transmitting data via the OTT connection 1650, in step 1608, the host 1602 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1606. In other embodiments, the user data is associated with a UE 1606 that shares data with the host 1602 without explicit human interaction. In step 1610, the host 1602 initiates a transmission carrying the user data towards the UE 1606. The host 1602 may initiate the transmission responsive to a request transmitted by the UE 1606. The request may be caused by human interaction with the UE 1606 or by operation of the client application executing on the UE 1606. The transmission may pass via the network node 1604, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1612, the network node 1604 transmits to the UE 1606 the user data that was carried in the transmission that the host 1602 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1614, the UE 1606 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1606 associated with the host application executed by the host 1602.

In some examples, the UE 1606 executes a client application which provides user data to the host 1602. The user data may be provided in reaction or response to the data received from the host 1602. Accordingly, in step 1616, the UE 1606 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input/output interface of the UE 1606. Regardless of the specific manner in which the user data was provided, the UE 1606 initiates, in step 1618, transmission of the user data towards the host 1602 via the network node 1604. In step 1620, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1604 receives user data from the UE 1606 and initiates transmission of the received user data towards the host 1602. In step 1622, the host 1602 receives the user data carried in the transmission initiated by the UE 1606.

One or more of the various embodiments improve the performance of OTT services provided to the UE 1606 using the OTT connection 1650, in which the wireless connection 1670 forms the last segment. More precisely, the teachings of these embodiments may improve the usefulness of ML model predictions sent between nodes and thereby provide benefits such as improved modelling, reduced erroneous modelling, reduced wasted transmissions (due to unusable ML model predictions being sent between nodes), and so on.

In an example scenario, factory status information may be collected and analyzed by the host 1602. As another example, the host 1602 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1602 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1602 may store surveillance video uploaded by a UE. As another example, the host 1602 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1602 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and/or transmitting data.

In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency, and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1650 between the host 1602 and UE 1606, in response to variations in the measurement results. The measurement procedure and/or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1602 and/or UE 1606. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1650 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1650 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1604. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency, and the like, by the host 1602. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or 'dummy' messages, using the OTT connection 1650 while monitoring propagation times, errors, etc.

Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and/or software needed to perform the tasks, features, functions, and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and/or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and/or by end users and a wireless network generally.

Some example embodiments of the present disclosure are as follows:

Group A Embodiments

Embodiment 1 : A method performed by a further network node for Machine Learning, ML, model uncertainty determination, the method comprising: receiving, from a network node, ML model uncertainty requirements; in response to the received ML model uncertainty requirements, determining whether or not to initiate transmission, to the network node, of ML model information; and where it is determined to initiate transmission of the ML model information, initiating transmission of the ML model information to the network node.

Embodiment 2: The method of embodiment 1, wherein the ML model uncertainty requirements comprise ML model prediction uncertainty value requirements for the further network node to satisfy when providing predictions to the network node.

Embodiment 3: The method of embodiment 2, wherein the ML model prediction uncertainty value requirements are received by the further network node as an uncertainty metric format, UMF.

Embodiment 4: The method of any of embodiments 2 and 3, wherein the ML model prediction uncertainty value requirements specify uncertainty information to be provided by the further network node.

Embodiment 5: The method of any of embodiments 2 to 4, wherein ML model prediction uncertainty value requirements comprise conditions under which the further network node should not provide ML model predictions as ML model information.

Embodiment 6: The method of any preceding embodiment, wherein the further network node determines to initiate transmission of ML model information when the further network node is able to satisfy the ML model uncertainty requirements, and wherein the method further comprises, prior to initiating transmission of ML model information to the network node, selecting by the further network node a ML model configuration that satisfies the ML model uncertainty requirements.

Embodiment 7: The method of embodiment 6 further comprising, prior to initiating transmission of ML model information to the network node, initiating training or retraining of a ML model such that the ML model uncertainty requirements are satisfied.

Embodiment 8: The method of embodiment 7, wherein the ML model training or retraining is performed by the further network node, or wherein the ML model training or retraining is performed by a ML model hosting node connected to the further network node. Embodiment 9: The method of any preceding embodiment further comprising, prior to receiving ML model uncertainty requirements from the network node, initiating transmission by the further network node of information on how ML model uncertainties are calculated to the network node.

Embodiment 10: The method of embodiment 9, wherein the information on how ML model uncertainties are calculated is provided in the form of capabilities in uncertainty metrics.

Embodiment 11 : The method of any of embodiments 9 and 10, wherein the information on how ML model uncertainties are calculated comprises information on one or more of: support for per-sample uncertainty estimates during inference; support for statistical uncertainty from training; support for including epistemic uncertainty in inference; and a complexity metric associated with one or more of the above support options.

Embodiment 12: The method of any preceding embodiment further comprising, when ML model information has been transmitted to the network node, receiving from the network node ML model performance feedback.

Embodiment 13: The method of embodiment 12, further comprising utilizing the ML model performance feedback in the retraining of ML models.

Embodiment 14: The method of any preceding embodiment, wherein the further network node is a User Equipment and wherein the ML model information comprises signal quality predictions for use in handover decisions.

Embodiment 15: The method of any of embodiments 1 to 13, wherein the further network node is a Radio Access Network, RAN, node, and wherein the ML model information comprises load predictions for the RAN node.

Group B Embodiments

Embodiment 16: A method performed by a network node for Machine Learning, ML, model uncertainty control, the method comprising: initiating transmission, to a further network node, of ML model uncertainty requirements; and performing, at the network node, ML modelling.

Embodiment 17: The method of embodiment 16, wherein the ML model uncertainty requirements comprise ML model prediction uncertainty value requirements for the further network node to satisfy when providing predictions to the network node.

Embodiment 18: The method of embodiment 17, wherein the ML model prediction uncertainty value requirements are sent to the further network node as an uncertainty metric format, UMF.

Embodiment 19: The method of any of embodiments 17 and 18 wherein the ML model prediction uncertainty value requirements specify uncertainty information to be provided by the further network node.

Embodiment 20: The method of any of embodiments 17 to 19, wherein ML model prediction uncertainty value requirements comprise conditions under which the further network node should not provide ML model predictions as ML model information.

Embodiment 21 : The method of any of embodiments 17 to 20 further comprising receiving, from the further network node, information on how ML model uncertainties are calculated prior to transmitting ML model uncertainty requirements to the further network node.

Embodiment 22: The method of embodiment 21, wherein the information on how ML model uncertainties are calculated is provided in the form of capabilities in uncertainty metrics. Embodiment 23: The method of any of embodiments 21 and 22, wherein the information on how ML model uncertainties are calculated comprises information on one or more of: support for per-sample uncertainty measurements during inference; support for statistical uncertainty from training; support for including epistemic uncertainty in inference; and a complexity metric associated with one or more of the above support options.

Embodiment 24: The method of any of embodiments 21 to 23 wherein the network node analyses the received information on how ML model uncertainties are calculated and determines to include in the ML model uncertainty requirements instructions to the further network node to not provide ML model predictions as ML model information.

Embodiment 25: The method of any of embodiments 21 to 23, further comprising receiving ML model information from the further network node, wherein the ML modelling is performed utilizing the ML model information received from the further network node.

Embodiment 26: The method of embodiment 25, wherein the ML model information received from the further network node relates to uncertainties on a prediction made by the further network node and comprises one or more of: information on types of uncertainty the prediction is subject to; information on levels of uncertainty relating to the prediction; whether the uncertainty is per-sample uncertainty; whether the uncertainty is statistical uncertainty from training; and whether the uncertainty includes epistemic uncertainty in inference.

Embodiment 27: The method of any of embodiments 25 and 26, wherein the network node utilizes the ML model information from the further network node during the ML modelling to: assign a weight to the prediction from the further network node; evaluate the uncertainty of any prediction made by the network node; determine whether or not to discard the prediction from the further network node; determine whether the prediction from the further network node can be used in inference calculations.

Embodiment 28: The method of any of embodiments 21 to 27 wherein, based on the ML model information received from the further network node, the network node initiates transmission to the further network node of ML model performance feedback.

Embodiment 29: The method of embodiment 28, wherein the ML model performance feedback comprises an instruction to perform further ML model training.

Embodiment 30: The method of any of embodiments 16 to 29, wherein the network node is a base station or core network node, and wherein the ML modelling comprises signal quality predictions for use in handover decisions.

Embodiment 31 : The method of any of embodiments 16 to 29, wherein the network node is a Radio Access Network, RAN, node, and wherein the ML modelling comprises load predictions.

Group C Embodiments

Embodiment 32: A further network node for ML model uncertainty determination, comprising: processing circuitry configured to cause the further network node to perform any of the steps of any of the Group A embodiments; and power supply circuitry configured to supply power to the processing circuitry. Embodiment 33: A network node for ML model uncertainty control, the network node comprising: processing circuitry configured to cause the network node to perform any of the steps of any of the Group B embodiments; power supply circuitry configured to supply power to the processing circuitry.

Embodiment 34: A further network node for ML model uncertainty determination, the further network node comprising: an antenna configured to send and receive wireless signals; radio front-end circuitry connected to the antenna and to processing circuitry, and configured to condition signals communicated between the antenna and the processing circuitry; the processing circuitry being configured to perform any of the steps of any of the Group A embodiments; an input interface connected to the processing circuitry and configured to allow input of information into the further network node to be processed by the processing circuitry; an output interface connected to the processing circuitry and configured to output information from the further network node that has been processed by the processing circuitry; and a battery or other power source connected to the processing circuitry and configured to supply power to the further network node.

Embodiment 35: A host configured to operate in a communication system to provide an over-the-top (OTT) service, the host comprising: processing circuitry configured to provide data; and a network interface configured to initiate transmission of the data to a cellular network for transmission to a further network node, wherein the further network node comprises a communication interface and processing circuitry, the communication interface and processing circuitry of the further network node being configured to perform any of the steps of any of the Group A embodiments to receive the data from the host.

Embodiment 36: The host of the previous embodiment, wherein the cellular network further includes a network node configured to communicate with the further network node to transmit the data to the further network node from the host.

Embodiment 37: The host of the previous 2 embodiments, wherein: the processing circuitry of the host is configured to execute a host application, thereby providing the data; and the host application is configured to interact with a client application executing on the further network node, the client application being associated with the host application.

Embodiment 38: A method implemented by a host operating in a communication system that further includes a network node and a further network node, the method comprising: providing user data for the further network node; and initiating a transmission carrying the user data to the further network node via a cellular network comprising the network node, wherein the further network node performs any of the operations of any of the Group A embodiments to receive the user data from the host.

Embodiment 39: The method of the previous embodiment, further comprising: at the host, executing a host application associated with a client application executing on the further network node to receive the user data from the further network node.

Embodiment 40: The method of the previous embodiment, further comprising: at the host, transmitting input data to the client application executing on the further network node, the input data being provided by executing the host application, wherein the user data is provided by the client application in response to the input data from the host application. Embodiment 41 : A host configured to operate in a communication system to provide an over-the-top (OTT) service, the host comprising: processing circuitry configured to provide data; and a network interface configured to initiate transmission of the data to a cellular network for transmission to a further network node, wherein the further network node comprises a communication interface and processing circuitry, the communication interface and processing circuitry of the further network node being configured to perform any of the steps of any of the Group A embodiments to transmit the data to the host.

Embodiment 42: The host of the previous embodiment, wherein the cellular network further includes a network node configured to communicate with the further network node to transmit the data from the further network node to the host.

Embodiment 43: The host of the previous 2 embodiments, wherein: the processing circuitry of the host is configured to execute a host application, thereby providing the data; and the host application is configured to interact with a client application executing on the further network node, the client application being associated with the host application.

Embodiment 44: A method implemented by a host configured to operate in a communication system that further includes a network node and a further network node, the method comprising: at the host, receiving user data transmitted to the host via the network node by the further network node, wherein the further network node performs any of the steps of any of the Group A embodiments to transmit the user data to the host.

Embodiment 45: The method of the previous embodiment, further comprising: at the host, executing a host application associated with a client application executing on the further network node to receive the user data from the further network node.

Embodiment 46: The method of the previous embodiment, further comprising: at the host, transmitting input data to the client application executing on the further network node, the input data being provided by executing the host application, wherein the user data is provided by the client application in response to the input data from the host application.

Embodiment 47: A host configured to operate in a communication system to provide an over-the-top (OTT) service, the host comprising: processing circuitry configured to provide data; and a network interface configured to initiate transmission of the data to a network node in a cellular network for transmission to a user equipment (UE), the network node having a communication interface and processing circuitry, the processing circuitry of the network node configured to perform any of the operations of any of the Group B embodiments to transmit the user data from the host to the UE.

Embodiment 48: The host of the previous embodiment, wherein: the processing circuitry of the host is configured to execute a host application that provides the user data; and the UE comprises processing circuitry configured to execute a client application associated with the host application to receive the transmission of user data from the host.

Embodiment 49: A method implemented in a host configured to operate in a communication system that further includes a network node and a further network node, the method comprising: providing data for the further network node; and initiating a transmission carrying the data to the further network node via a cellular network comprising the network node, wherein the network node performs any of the operations of any of the Group B embodiments to transmit the data from the host to the further network node.

Embodiment 50: The method of the previous embodiment, further comprising, at the network node, transmitting the data provided by the host for the further network node.

Embodiment 51 : The method of any of the previous 2 embodiments, wherein the data is provided at the host by executing a host application that interacts with a client application executing on the further network node, the client application being associated with the host application.

Embodiment 52: A communication system configured to provide an over-the-top service, the communication system comprising: a host comprising: processing circuitry configured to provide user data for a user equipment (U E), the user data being associated with the over-the-top service; and a network interface configured to initiate transmission of the user data toward a cellular network node for transmission to the UE, the network node having a communication interface and processing circuitry, the processing circuitry of the network node configured to perform any of the operations of any of the Group B embodiments to transmit the user data from the host to the UE.

Embodiment 53: The communication system of the previous embodiment, further comprising: the network node; and/or the user equipment.

Embodiment 54: A host configured to operate in a communication system to provide an over-the-top (OTT) service, the host comprising: processing circuitry configured to initiate receipt of user data; and a network interface configured to receive the user data from a network node in a cellular network, the network node having a communication interface and processing circuitry, the processing circuitry of the network node configured to perform any of the operations of any of the Group B embodiments to receive the user data from a user equipment (UE) for the host.

Embodiment 55: The host of the previous 2 embodiments, wherein: the processing circuitry of the host is configured to execute a host application, thereby providing the user data; and the host application is configured to interact with a client application executing on the UE, the client application being associated with the host application.

Embodiment 56: The host of the any of the previous 2 embodiments, wherein the initiating receipt of the user data comprises requesting the user data.

Embodiment 57: A method implemented by a host configured to operate in a communication system that further includes a network node and a user equipment (UE), the method comprising: at the host, initiating receipt of user data from the UE, the user data originating from a transmission which the network node has received from the UE, wherein the network node performs any of the steps of any of the Group B embodiments to receive the user data from the UE for the host.

Embodiment 58: The method of the previous embodiment, further comprising at the network node, transmitting the received user data to the host.

Those skilled in the art will recognize improvements and modifications to the embodiments of the present disclosure. All such improvements and modifications are considered within the scope of the concepts disclosed herein.

Claims

1 . A method performed by a first node, the method comprising: sending (Fig. 4, step 120), to a second node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions.

2. The method of claim 1 wherein the uncertainty information comprises information that indicates one or more types of uncertainty the one or more output predictions are subject to.

3. The method of claim 2 wherein the uncertainty information further comprises information about the one or more types of uncertainty the one or more output predictions are subject to.

4. The method of any of claims 1 to 3 wherein the uncertainty information comprises: information that indicates whether the uncertainty information is per-sample uncertainty during inference; information that indicates whether the uncertainty information is statistical uncertainty from training of the first ML model; information that indicates whether the uncertainty information comprises epistemic uncertainty in inference; or any combination thereof.

5. The method of any of claims 1 to 4 wherein the uncertainty information comprises, per-sample uncertainty information for each output prediction from the first ML model, the per-sample uncertainty information being dependent on respective inputs to the first ML model.

6. The method of any of claims 1 to 5 wherein an output prediction from the first model is a point estimate, and the uncertainty information comprises an uncertainty value for the point estimate.

7. The method of any of claims 1 to 5 wherein the uncertainty information comprises or is comprised in a probabilistic prediction for a respective output prediction, and the output prediction can be derived from the probabilistic prediction.

8. The method of any of claims 1 to 4 wherein the uncertainty information comprises information that indicates that the uncertainty information comprises information about a statistical uncertainty of the one or more output predictions as a result of training of the first ML model.

9. The method of claim 8 wherein the information about the statistical uncertainty of the one or more output predictions as a result of training of the first ML model comprises information about a homoscedastic uncertainty of the one or more output predictions.

10. The method of claim 8 wherein the information about the statistical uncertainty of the one or more output predictions as a result of training of the first ML model comprises information about a heteroscedastic uncertainty of the one or more output predictions.

11. The method of any of claims 1 to 10 further comprising: receiving (Fig. 4, step 110; 502), from the second node, information that indicates one or more ML model uncertainty requirements.

12. The method of claim 11 further comprising: determining (504) whether or not to initiate the sending of the one or more output predictions from the first ML model to the second node, based on the ML model uncertainty requirements received from the second node; wherein sending (Fig. 4, step 120) the one or more output predictions together with the uncertainty information about the one or more output predictions comprises sending (Fig. 4, step 120) the one or more output predictions together with the uncertainty information about the one or more output predictions to the second node in response to determining to initiate the sending of the one or more output predictions from the first ML model to the second node.

13. The method of claim 12 further comprising retraining (Fig. 4, step 115) the first ML model based on the information that indicates the one or more ML model uncertainty requirements received from the second node.

14. The method of any of claims 11 to 13 wherein the ML model uncertainty requirements comprise ML model prediction uncertainty value requirements for the first node to satisfy in regard to the one or more output predictions sent to the second node.

15. The method of claim 14 wherein the ML model prediction uncertainty value requirements specify uncertainty information to be provided by the first node.

16. The method of claim 14 wherein the ML model prediction uncertainty value requirements comprise conditions under which the first node should not provide ML model output predictions.

17. The method of any of claims 1 to 16 further comprising sending (Fig. 4, step 100), to the second node, capability information that indicates one or more capabilities of the first node related to computing and/or reporting uncertainty of the one or more output predictions of the first ML model.

18. The method of claim 17 wherein the capability information comprises: information that indicates that the first node supports computing and/or reporting of per-sample prediction uncertainty during inference; information that indicates that the first node supports computing and/or reporting of statistical uncertainty from training; information that indicates that the first node supports computing and/or reporting of epistemic uncertainty in inference; or any combination thereof.

19. The method of any of claims 1 to 18 wherein the first node and/or the second node is a network node of a cellular communications system.

20. The method of any of claims 1 to 18 wherein the first node is a User Equipment, UE, and the second node is a network node in a Radio Access Network, RAN, wherein the one or more output predictions comprise one or more signal quality predictions for use in a handover decision.

21. The method of any of claims 1 to 18 wherein the first node is a network node in a Radio Access Network, RAN, and the one or more output predictions comprise one or more load predictions and/or one or more energysaving predictions for the network node.

22. A first node adapted to: send (Fig. 4, step 120), to a second node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions.

23. The first node of claim 22 further adapted to perform the method of any of claims 2 to 21 .

24. A first node comprising processing circuitry configured to cause the first node to: send (Fig. 4, step 120), to a second node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions.

25. The first node of claim 24 wherein the processing circuitry is further configured to cause the first node to perform the method of any of claims 2 to 21 .

26. A method performed by a second node, the method comprising: receiving (Fig. 4, step 120), from a first node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions; and using (Fig. 4, step 125 or 135) the one or more output predictions and uncertainty information in combination with one or more other inputs to perform one or more actions related to operation of a radio network.

27. The method of claim 26 wherein the uncertainty information comprises information that indicates one or more types of uncertainty the one or more one or more output predictions are subject to.

28. The method of claim 27 wherein the uncertainty information further comprises information about the one or more types of uncertainty the one or more one or more output predictions are subject to.

29. The method of any of claims 26 to 28 wherein the uncertainty information comprises: information that indicates whether the uncertainty information is per-sample uncertainty during inference; information that indicates whether the uncertainty information is statistical uncertainty from training of the first ML model; information that indicates whether the uncertainty information comprises epistemic uncertainty in inference; or any combination thereof.

30. The method of any of claims 26 to 29 wherein the uncertainty information comprises, per-sample uncertainty information for each output prediction from the first ML model, the per-sample uncertainty information being dependent on respective inputs to the first ML model.

31 . The method of any of claims 26 to 30 wherein an output prediction from the first model is a point estimate, and the uncertainty information comprises an uncertainty value for the point estimate.

32. The method of any of claims 26 to 30 wherein the uncertainty information comprises a probabilistic prediction for a respective output prediction, and the output prediction can be derived from the probabilistic prediction.

33. The method of any of claims 26 to 29 wherein the uncertainty information comprises information that indicates that the uncertainty information comprises information about a statistical uncertainty of the one or more output predictions as a result of training of the first ML model.

34. The method of claim 33 wherein the information about the statistical uncertainty of the one or more output predictions as a result of training of the first ML model comprises information about a homoscedastic uncertainty of the one or more output predictions.

35. The method of claim 33 wherein the information about the statistical uncertainty of the one or more output predictions as a result of training of the first ML model comprises information about a heteroscedastic uncertainty of the one or more output predictions.

36. The method of any of claims 26 to 35 further comprising: sending (Fig. 4, step 110; step 502), to the first node, information that indicates one or more ML model uncertainty requirements.

37. The method of any of claims 26 to 36 further comprising receiving (Fig. 4, step 100), from the first node, capability information that indicates one or more capabilities of the first node related to computing and/or reporting uncertainty of the one or more output predictions of the first ML model.

38. The method of claim 37 wherein the capability information comprises: information that indicates that the first node supports computing and/or reporting per-sample prediction uncertainty during inference; information that indicates that the first node supports computing and/or reporting statistical uncertainty from training; information that indicates that the first node supports computing and/or reporting epistemic uncertainty in inference; or any combination thereof.

39. The method of any of claims 26 to 38 wherein the first node and/or the second node is a network node of a cellular communications system.

40. A second node adapted to:receive (Fig. 4, step 120), from a first node, one or more output predictions from a first machine learning, ML, model together with uncertainty information about the one or more output predictions; and use (Fig. 4, step 125 or 135) the one or more output predictions and uncertainty information in combination with one or more other inputs to perform one or more actions related to operation of a radio network.

41 . The second node of claim 40 further adapted to perform the method of any of claims 27 to 39.

42. A second node comprising processing circuitry configured to cause the second node to: receive (Fig. 4, step 120), from a first node, one or more output predictions from a first machine learning,

ML, model together with uncertainty information about the one or more output predictions; and use (Fig. 4, step 125 or 135) the one or more output predictions and uncertainty information in combination with one or more other inputs to perform one or more actions related to operation of a radio network.

43. The second node of claim 42 wherein the processing circuitry is further configured to cause the second node to perform the method of any of claims 27 to 39.