EP4681119A1

EP4681119A1 - Probabilistic client selection and resource allocation for federated learning

Info

Publication number: EP4681119A1
Application number: EP23715954.6A
Authority: EP
Inventors: Wen Xu; Ben Liang; Gary Boudreau; Hamza SOKUN
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2023-03-23
Filing date: 2023-03-23
Publication date: 2026-01-21
Also published as: WO2024194679A1

Abstract

A method implemented in a server configured to communicate with a plurality of clients in a wireless communication system is provided. An aggregate machine learning model for federated learning is initialized, and a federated learning is performed, with each round of the federated learning including determining an optimal subset of the plurality of clients, causing transmission, to at least the optimal subset of the plurality of clients, of a first indication of the current state of the aggregate machine learning model, receiving, from the optimal subset of the plurality of clients, a plurality of second indications corresponding to local machine learning models associated with the optimal subset of the plurality of clients, and updating the aggregate machine learning model by aggregating the local machine learning models.

Description

PROBABILISTIC CLIENT SELECTION AND RESOURCE ALLOCATION FOR FEDERATED LEARNING TECHNICAL FIELD The present disclosure relates to wireless communications, and in particular, to configurations for supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. BACKGROUND The Third Generation Partnership Project (3GPP) has developed and is developing standards for Fourth Generation (4G) (also referred to as Long Term Evolution (LTE)) and Fifth Generation (5G) (also referred to as New Radio (NR)) wireless communication systems. Such systems provide, among other features, broadband communication between network nodes, such as base stations, and mobile wireless devices (WD), as well as communication between network nodes and between WDs. The 3GPP is also developing standards for Sixth Generation (6G) wireless communication networks. Federated learning (FL), which is a distributed machine learning paradigm which operates on decentralized data, has gained attention from both academia and the industry recently. In some existing FL configurations, raw data is privately held by each client (e.g., in a distributed system or network including a plurality of clients in communication with one or more servers) without the data being shared with other clients. This is different from the classic setting of distributed learning, e.g., in data centers where each node may have access to all the data through data sharing. A goal of FL is to collaboratively train a machine learning model while maintaining the privacy of data on each client and by only transmitting models and extra control variates without transmitting the raw data, e.g., from client to server or from client to client. For example, a cross-device setting may include an edge server or base station (BS) which coordinates the training of one global model with decentralized data on many clients, which may include, e.g., mobile or IoT devices, other network nodes or base stations, etc. Although FL enjoys many advantages such as preserving data privacy, utilizing resources of end devices, and possessing the flexibility of model personalization, FL may experience challenges such as system and statistical heterogeneity, which may have a negative impact on the efficiency and effectiveness of the training. As an example, statistical heterogeneity may be roughly divided into three aspects: the size of local data, non-identical data distributions among clients, and underlying structures of data. In some systems, data within each client may be independent and identically distributed (IID), but the data distributions across clients may be non-identical, and the size of samples of different clients may vary significantly. The system heterogeneity lies in multiple system aspects such as communication conditions, computing resources, and the energy consumption. System heterogeneity and statistical heterogeneity make the client sampling and resource allocation of FL challenging in the real world. If clients are blindly chosen, for example, the data quality may not be good, and if resources are casual allocations, their utility may be low. To address some of these deficiencies, some existing systems employ a randomized client selection paradigm, where clients may be selected based on some probabilities. Some existing systems provide various techniques for selecting clients and allocating resources at each round in the training phase of FL. Existing solutions for client sampling and/or resource allocation, however, suffer from a number of drawbacks. Client Sampling for FL For example, there are existing solutions which utilize an optimization problem whose objective is to maximize the number of selected clients and includes constraints such as time constraints, round time, etc. A greedy selection strategy may be used that may result in a high-performance model in a significantly shorter time than some other systems, but without any theoretical proof of correctness, and which may fail to account for dynamic scenarios such as fluctuations in updating and uploading time. In another example existing system, a tier-based Federated Learning System is used which divides clients into tiers based on performance, and which only selects clients of the same tier in each round. In another example existing system, an adaptive tier selection algorithm, a meta-algorithm of geometrically increasing client participation may be used to tackle stragglers. For a strongly convex objective, it may be shown, with theoretical analysis, that such a method may outperform some other existing FL methods. However, in all the aforementioned approaches, the sampling strategies for client participation are deterministic based on the solution of some optimization problems or in a greedy manner. In turn, it is possible to only select a subset of all possible clients which, may exclude the chance to let the model get access to the entire data of all clients. Probabilistic Client Sampling for FL In some existing systems, the term “federated learning” may refer to a server choosing a fraction of all clients for participation at each round, e.g., in a uniformly random way. This randomized approach utilizes the local data of all available clients with almost complete certainty, e.g., if the training rounds are large enough. Instead of implicitly using probabilistic client sampling in this “vanilla” FL, optimal client sampling in FL may explicitly use the norms of the gradients, or their approximations, which measure the importance of the data on clients, to sample the clients in a probabilistic manner, adopting the idea from the importance sampling of training data to minimize the variance of the stochastic gradient for machine learning. For example, the probabilities of client selection may be based on the norms of updates or their approximations to minimize the variance of the gradient estimator. One example theoretical analysis for convex and non-convex objectives has shown that the performance of optimal client sampling method may be close to full participation and superior to the baseline, where clients are selected uniformly at random. However, existing methods with respect to learning and the number of global rounds may fail to take into account realistic factors such as communication conditions and computation resources. Taking into account wireless communications, for example, a new optimization problem may be formed and solved so as to minimize the convergence time of FL and at the same time optimize the performance of FL. However, employing a probabilistic client sampling approach may require one client to always be connected to the server at each communication round, which may not be ideal, as the channel condition of that client can fluctuate and even make the client itself a straggler. A framework for measuring the performance of FL over wireless networks has been proposed, e.g., where sampling strategies such as random scheduling (RS), round robin (RR), and proportional fair (PF) have been considered. Work focusing on minimizing the expected wall-clock time, e.g., with constraints on convergence and sampling probabilities has been undertaken. This takes into consideration both the statistical and system heterogeneity, and provides convergence bounds for arbitrary sampling probabilities for smooth and convex objects. However, sampling probabilities may need to be fixed in all rounds, which may be insufficient to capture the dynamics of system conditions. Joint Client Sampling and Resource allocation for FL The problem of resource allocation for FL has been extensively studied in the communication community, often with the joint consideration of client sampling. For example, in one study, an optimization problem has been proposed and solved to jointly design the FL algorithm and the resource allocation in wireless networks with considerations of delay requirements, energy consumption, transmit power, and packet error rate. Another joint learning and resource allocation optimization problem has been derived and solved, where the trade-off between the wall clock time for FL convergence and resource allocation such as transmission power and CPU frequencies may be captured in the formation of the optimization problem. However, in these existing approaches, the client sampling and resource allocation may be deterministic from the solutions of some optimizations and may be fixed through rounds, which fail to capture the dynamics of system conditions. Recent work attempts to address this issue by combining probabilistic client sampling and Lyapunov optimization, formulating and solving a novel stochastic optimization where the objective is the time average of a combination of sampling bound and communication time and the constraints are time average and peak of transmission power as well as control actions, i.e., sampling probabilities, under a Lyapunov drift-plus-penalty framework. However, this formulation does not consider different sizes of samples on clients and the actual gradients of the loss function, which are significant in the performance of federated learning, and further, it has no control over the expected number of selected clients, which may be essential to some real-world systems that can only handle up to a certain number of clients. Furthermore, this existing work only deals with power allocation for communications. Thus, existing systems lack adequate configurations for supporting federated learning in a wireless communication system. SUMMARY Some embodiments advantageously provide methods, systems, and apparatuses for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system. Embodiments of the present disclosure may overcome one or more of the above-described drawbacks of existing solutions, for example, by (1) accounting for the gradients of the loss function in the mathematical analysis/computation and/or in the optimization solution, which further improves the performance, especially when the distributions across clients are different; and/or (2) providing algorithms and process flows which are practical and adaptable for real world solutions, e.g., by providing control over the number of selected clients in an expectation, which may cause the optimization problem to become biconvex, which may in turn require different techniques to solve as compared to existing systems, as described herein; and/or (3) enabling valuable design flexibility for resource allocation, e.g., by allowing the allocation of multiple types of resources including but not limited to transmit power. Embodiments of the present disclosure provide configurations for supporting joint optimization of probabilistic client sampling and resource allocation in FL. A stochastic network optimization problem solution may be performed, e.g., by a server in a FL context, by combining the mathematical term/information containing the sampling probabilities of clients from the convergence bound of a smooth global loss function and the term/information of the costs of a resource allocation as the objective. The constraints may include both long-term expectation and short-term for resource allocation and control actions with respect to the sampling probabilities. Utilizing the solutions from the per-time-slot opportunistic scheduling problems, at least two algorithms may be supported by embodiments of the present disclosure: (i) the Probabilistic Client Selection and Resource Allocation Algorithm (PCSRA), and (ii) the Dynamic Probabilistic Client Selection and Resource Allocation Algorithm (Dynamic-PCSRA) under a probabilistic sampling framework. Frameworks and algorithms are described with respect to embodiments herein, including concrete examples for joint optimization of client sampling and power allocation for communications. In experiments applying embodiments of the present disclosure to simulated datasets and network environments, the embodiments of the present disclosure may be shown to outperform baseline solutions (e.g., representing some kinds of existing systems) of separate optimizations and other joint optimizations. Embodiments of the present disclosure may support configurations for a new convergence bound technique for non-convex loss functions in a FL framework with probabilistic client sampling, which may properly account for statistical heterogeneity. Using this bound technique, embodiments of the present disclosure may support joint optimization of online probabilistic client sampling and resource allocation, e.g., to minimize the time average of a combination of terms of convergence and costs of allocations. Embodiments may be suitable for allocations of multiple resources, and may enable flexible control over the expected number of selected clients, which may be configured based on system requirements, capabilities, and tasks. In some embodiments, a Lyapunov optimization may be used to compute the aforementioned stochastic optimization problem. The non-convex per-slot problem may be computed, e.g., by decomposing it into two convex subproblems which may be solved and/or computed efficiently (e.g., individually), and which may provide one or more performance guarantees for the various algorithms, as described herein. Embodiments of the present disclosure may support one or more versions of online algorithms (e.g., PCSRA and Dynamic-PCSRA), based on the joint optimization problem, e.g., with an estimation of the upper bounds of norms of gradients, or with the actual per-round norms of stochastic gradients. There may be some trade-offs between costs and resources needed for communication and for computation, among the various embodiments of the present disclosure, as described herein. Several example implementations are described with respect to embodiments of the present disclosure, including, e.g., joint optimization of learning and power allocation for communications. Embodiments of the present disclosure may also be applied to many other use cases beyond the examples provided herein. A probabilistic client sampling configuration may be agnostic to the particular type of learning task. Embodiments of the present disclosure may perform, e.g., a task of image classification, but any machine learning task such as object detection, action recognition, sentiment analysis, speech recognition, etc., may be applicable to embodiments of the present disclosure. Further, embodiments of the present disclosure may be used to optimize resources of a variety of resource types, such as power, bandwidth, radio channel resources (e.g., time domain resources, frequency domain resources, etc.), CPU frequency, etc. Embodiments of the present disclosure may provide one or more advantages over existing systems: - Embodiments of the present disclosure may support jointly optimizing for probabilistic client selection and resource allocation. For example, taking into account statistical heterogeneity, such as different sizes of data and importance of data on clients and the costs and constraints of resource allocation, may result in improved efficiency, optimization, resource usage, etc. - By imposing a constraint on expected number of selected clients, embodiments of the present disclosure may enable flexible control over the expected number of clients, which may be configurable (e.g., by network users or administrators) in order to tailor the embodiments for particular tasks and system specifications. - A Dynamic-PCSRA strategy, as described herein with respect to some embodiments, may use actual norms of gradients to measure the importance/quality of data samples at each global round in federated learning, mimicking the optimal client sampling for FL, and satisfying long-term and short-term constraints. - Embodiments of the present disclosure may provide solutions for a variety of different real-world applications. As one example, embodiments may provide support for the joint optimization of learning and power allocation for communications systems where there may be some system heterogeneity such as channel conditions. In some embodiments, it is not necessary for the system to know the distribution of the heterogeneity in channel conditions. - Embodiments of the present disclosure may be applicable to a “FedLoc” use case, for example, where the task is to collaboratively build accurate location services, e.g., a machine learning model trained using FL. For example, for solving a static localization problem, the input of a machine learning model contains the radio features at different locations, and the output is the position where the radio feature is measured. Many mobile terminals or wireless devices of different computational capacities that collect data for particular areas may be configured to collaboratively train a model for the global estimate of the function mapping from radio features to a position/location. With properly defined constraints, a FedLoc application may be applied to embodiments of the present disclosure, e.g., by performing probabilistic client sampling while satisfying the resource constraints defined by the application. As described herein, simulated experiments may demonstrate that sampling strategies according to embodiments of the present disclosure may outperform a baseline of separate optimization strategies and other joint optimization strategies. When it is assumed that base stations (e.g., network nodes) and/or servers (e.g., cloud-based nodes, core nodes, virtual servers, etc.) coordinate the training of a global model, then, wireless device-specific information available at the base stations (network nodes), such as CQI reports or the estimated effective SINRs used in the link adaptation or wireless device-locations, may also be leveraged to further improve the performance of the proposed client sampling strategies. According to a first aspect of the present disclosure, a server configured to communicate with a plurality of clients (e.g., wireless devices, network nodes, etc.) in a wireless communication system is provided. The server is configured to initialize an aggregate machine learning model for federated learning. The server is configured to perform a federated learning on the aggregate machine learning model with the plurality of clients for a plurality of rounds, where a round of the federated learning includes determining an optimal subset of the plurality of clients, transmitting and/or causing transmission, to at least the optimal subset of the plurality of clients, of a first indication of a current state of the aggregate machine learning model, receiving, from the optimal subset of the plurality of clients, a plurality of second indications corresponding to local machine learning models associated with the optimal subset of the plurality of clients, and updating the aggregate machine learning model by aggregating the local machine learning models. According to one or more embodiments of this aspect, the server is further configured to transmit and/or cause transmission of an initial configuration indication to the plurality of clients, where the initial configuration indication includes at least one of a local learning rate, and a number of local steps. According to one or more embodiments of this aspect, the server is further configured to initialize the aggregate machine learning model by initializing at least one virtual queue, and perform the federated learning by updating the at least one virtual queue based on the local machine learning models. According to one or more embodiments of this aspect, the server is further configured to, for each round of the federated learning, transmit and/or cause transmission of, to all of the plurality of clients, the first indication of the current state of the aggregate machine learning model, for each round of the federated learning, receive, responsive to the first indication, a plurality of third indications of a plurality of stochastic gradients associated with the plurality of clients, and for each round of the federated learning, the determining of the corresponding optimal subset of the plurality of clients is based on the plurality of stochastic gradients. According to one or more embodiments of this aspect, the determining of the optimal subset of the plurality of clients is based on a convex optimization of at least one of radio channel resources associated with the plurality of clients, processing resources associated with the plurality of clients, and power resources associated with the plurality of clients. According to one or more embodiments of this aspect, the server is further configured to receive radio measurement data associated with at least one client of the plurality of clients, and predict a location of the at least one client based on the radio measurement data using the aggregate machine learning model. According to another aspect of the present disclosure, a method in a server configured to communicate with a plurality of clients (e.g., wireless devices, network nodes, etc.) in a wireless communication system is provided. The method includes initializing an aggregate machine learning model for federated learning. The method includes performing a federated learning on the aggregate machine learning model with the plurality of clients for a plurality of rounds, where a round of the federated learning includes determining an optimal subset of the plurality of clients, transmitting and/or causing transmission, to at least the optimal subset of the plurality of clients, of a first indication of a current state of the aggregate machine learning model, receiving, from the optimal subset of the plurality of clients, a plurality of second indications corresponding to local machine learning models associated with the optimal subset of the plurality of clients, and updating the aggregate machine learning model by aggregating the local machine learning models. According to one or more embodiments of this aspect, the method further includes transmitting and/or causing transmission of an initial configuration indication to the plurality of clients, where the initial configuration indication includes at least one of a local learning rate, and a number of local steps. According to one or more embodiments of this aspect, the method further includes initializing the aggregate machine learning model by initializing at least one virtual queue, and performing the federated learning by updating the at least one virtual queue based on the local machine learning models. According to one or more embodiments of this aspect, the method further includes, for each round of the federated learning, transmitting and/or causing transmission of, to all of the plurality of clients, the first indication of the current state of the aggregate machine learning model, for each round of the federated learning, receiving, responsive to the first indication, a plurality of third indications of a plurality of stochastic gradients associated with the plurality of clients, and for each round of the federated learning, the determining of the corresponding optimal subset of the plurality of clients is based on the plurality of stochastic gradients. According to one or more embodiments of this aspect, the determining of the optimal subset of the plurality of clients is based on a convex optimization of at least one of radio channel resources associated with the plurality of clients, processing resources associated with the plurality of clients, and power resources associated with the plurality of clients. According to one or more embodiments of this aspect, the method further includes receiving radio measurement data associated with at least one client of the plurality of clients, and predicting a location of the at least one client based on the radio measurement data using the aggregate machine learning model. According to another aspect of the present disclosure, a client (e.g., a wireless device, network node, etc.) configured to communicate with a server in a wireless communication system including a plurality of additional clients in communication with the server is provided. The client is configured to at least one of receive, store, and determine a dataset for training a local machine learning model. The client is configured to perform a federated learning for an aggregate machine learning model with the plurality of additional clients for a plurality of rounds, where a round of the federated learning includes, receiving, from the server, a first indication of a current state of the aggregate machine learning model, training, in parallel with an optimal subset of the plurality of additional clients, a local machine learning model based on the current state of the aggregate machine learning model and the dataset, and transmitting and/or causing transmission of, to the server, a second indication corresponding to the local machine learning model. According to one or more embodiments of this aspect, the client is further configured to receive an initial configuration indication from the server, where the initial configuration indication includes at least one of a local learning rate, and a number of local steps, and the training of the local machine learning model is based on the local learning rate and the number of local steps. According to one or more embodiments of this aspect, the client is further configured to, for each round of the federated learning, receive the first indication of the current state of the aggregate machine learning model, for each round of the federated learning, determine a stochastic gradient based on the local machine learning model, and for each round of the federated learning, cause transmission, responsive to the first indication, of a third indication of the stochastic gradient, where the optimal subset of the plurality of clients is determined based on the stochastic gradient. According to one or more embodiments of this aspect, the optimal subset of the plurality of clients is determined based on a convex optimization of at least one of radio channel resources associated with the plurality of clients, processing resources associated with the plurality of clients, and power resources associated with the plurality of clients. According to one or more embodiments of this aspect, the client is further configured to cause transmission, to the server, of radio measurement data associated with at least one client of the plurality of clients, and receive a location prediction based on the radio measurement data and the aggregate machine learning model. According to another aspect of the present disclosure, a method in a client (e.g., a wireless device, network node, etc.) configured to communicate with a server in a wireless communication system including a plurality of additional clients in communication with the server is provided. The method includes at least one of receiving, storing, and determining a dataset for training a local machine learning model The method includes performing a federated learning for an aggregate machine learning model with the plurality of additional clients for a plurality of rounds, where a round of the federated learning includes, receiving, from the server, a first indication of a current state of the aggregate machine learning model, training, in parallel with an optimal subset of the plurality of additional clients, a local machine learning model based on the current state of the aggregate machine learning model and the dataset, and transmitting and/or causing transmission of, to the server, a second indication corresponding to the local machine learning model. According to one or more embodiments of this aspect, the method further includes receiving an initial configuration indication from the server, where the initial configuration indication includes at least one of a local learning rate, and a number of local steps, and the training of the local machine learning model is based on the local learning rate and the number of local steps. According to one or more embodiments of this aspect, the method further includes, for each round of the federated learning, receiving the first indication of the current state of the aggregate machine learning model, for each round of the federated learning, determining a stochastic gradient based on the local machine learning model, and for each round of the federated learning, transmitting and/or causing transmission of, responsive to the first indication, a third indication of the stochastic gradient, where the optimal subset of the plurality of clients is determined based on the stochastic gradient. According to one or more embodiments of this aspect, the optimal subset of the plurality of clients is determined based on a convex optimization of at least one of radio channel resources associated with the plurality of clients, processing resources associated with the plurality of clients, and power resources associated with the plurality of clients. According to one or more embodiments of this aspect, the method further includes transmitting and/or causing transmission of, to the server, radio measurement data associated with at least one client of the plurality of clients, and receive a location prediction based on the radio measurement data and the aggregate machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS A more complete understanding of the present embodiments, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein: FIG. 1 is a schematic diagram of an example network architecture illustrating a communication system connected via an intermediate network to a host computer according to the principles in the present disclosure; FIG. 2 is a block diagram of a host computer communicating via a network node with a wireless device and a server over an at least partially wireless connection according to some embodiments of the present disclosure; FIG. 3 is a flowchart illustrating example methods implemented in a communication system including a host computer, server, a network node and a wireless device for executing a client application at a wireless device according to some embodiments of the present disclosure; FIG. 4 is a flowchart illustrating example methods implemented in a communication system including a host computer, server, a network node and a wireless device for receiving user data at a wireless device according to some embodiments of the present disclosure; FIG. 5 is a flowchart illustrating example methods implemented in a communication system including a host computer, server, a network node and a wireless device for receiving user data from the wireless device at a host computer according to some embodiments of the present disclosure; FIG. 6 is a flowchart illustrating example methods implemented in a communication system including a host computer, server, a network node and a wireless device for receiving user data at a host computer according to some embodiments of the present disclosure; FIG. 7 is a flowchart of an example process in a server for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system according to some embodiments of the present disclosure; FIG. 8 is a flowchart of an example process in a client (e.g., a network node and/or wireless device) for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system according to some embodiments of the present disclosure; FIG. 9 is a flowchart of another example process for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system according to some embodiments of the present disclosure; FIG. 10 is a flowchart of another example process for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system according to some embodiments of the present disclosure; FIG. 11 is a flowchart of another example process for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system according to some embodiments of the present disclosure; FIG. 12 is a series of graphs illustrating an example set of simulated optimization outputs, according to one or more embodiments of the present disclosure; FIG. 13 is a series of graphs illustrating another example set of simulated optimization outputs, according to one or more embodiments of the present disclosure; FIG. 14 is a series of graphs illustrating another set of example simulated optimization outputs, according to one or more embodiments of the present disclosure; and FIG. 15 is a series of graphs illustrating another example set of simulated optimization outputs, according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION Before describing in detail example embodiments, it is noted that the embodiments reside primarily in combinations of apparatus components and processing steps related to configurations for supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. Accordingly, components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. Like numbers refer to like elements throughout the description. As used herein, relational terms, such as “first” and “second,” “top” and “bottom,” and the like, may be used solely to distinguish one entity or element from another entity or element without necessarily requiring or implying any physical or logical relationship or order between such entities or elements. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. In embodiments described herein, the joining term, “in communication with” and the like, may be used to indicate electrical or data communication, which may be accomplished by physical contact, induction, electromagnetic radiation, radio signaling, infrared signaling or optical signaling, for example. One having ordinary skill in the art will appreciate that multiple components may interoperate and modifications and variations are possible of achieving the electrical and data communication. In some embodiments described herein, the term “coupled,” “connected,” and the like, may be used herein to indicate a connection, although not necessarily directly, and may include wired and/or wireless connections. The term “network node” used herein can be any kind of network node comprised in a radio network which may further comprise any of base station (BS), radio base station, base transceiver station (BTS), base station controller (BSC), radio network controller (RNC), g Node B (gNB), evolved Node B (eNB or eNodeB), Node B, multi-standard radio (MSR) radio node such as MSR BS, multi-cell/multicast coordination entity (MCE), integrated access and backhaul (IAB) node, relay node, donor node controlling relay, radio access point (AP), transmission points, transmission nodes, Remote Radio Unit (RRU) Remote Radio Head (RRH), a core network node (e.g., mobile management entity (MME), self-organizing network (SON) node, a coordinating node, positioning node, MDT node, etc.), an external node (e.g., 3rd party node, a node external to the current network), nodes in distributed antenna system (DAS), a spectrum access system (SAS) node, an element management system (EMS), etc. The network node may also comprise test equipment. The term “radio node” used herein may be used to also denote a wireless device (WD) such as a wireless device (WD) or a radio network node. The terms “clients” and “server” may be used herein to describe embodiments of the present disclosure. A server may be generally any device which coordinates and/or aggregates machine learning models in a federated learning context, while a client may be generally any device which performs local training of local machine learning models and which is coordinated by a server in a federated learning context. In some embodiments, the non-limiting terms wireless device (WD) or a user equipment (UE) are used interchangeably. The WD herein can be any type of wireless device capable of communicating with a network node or another WD over radio signals, such as wireless device (WD). The WD may also be a radio communication device, target device, device to device (D2D) WD, machine type WD or WD capable of machine to machine communication (M2M), low-cost and/or low-complexity WD, a sensor equipped with WD, Tablet, mobile terminals, smart phone, laptop embedded equipped (LEE), laptop mounted equipment (LME), USB dongles, Customer Premises Equipment (CPE), an Internet of Things (IoT) device, or a Narrowband IoT (NB-IOT) device, etc. Also, in some embodiments the generic term “radio network node” is used. It can be any kind of a radio network node which may comprise any of base station, radio base station, base transceiver station, base station controller, network controller, RNC, evolved Node B (eNB), Node B, gNB, Multi-cell/multicast Coordination Entity (MCE), IAB node, relay node, access point, radio access point, Remote Radio Unit (RRU) Remote Radio Head (RRH). For example, as used herein, with respect to the context of federated learning, a network node may be a type of client, or may be a type of server, depending on the use case. For example, in some contexts, according to some embodiments, a network node may coordinate training of local machine learning models by one or more WDs in communication with the network node, in which case, the network node may be the “server” and the WDs may be the “clients”. In another context, according to some embodiments, a cloud node (e.g., a core node, orchestrator node, cloud-based server, virtual node, host computer, etc.) may coordinate a plurality of network nodes to perform training of local machine learning models for federated learning, where the cloud node aggregates the local machine learning models into an aggregate model. In this case, the cloud node may be the server, and the network nodes may be the client. In another context, according to some embodiments, a cloud node (e.g., a core node, orchestrator node, cloud-based server, virtual node, host computer, etc.) may coordinate (e.g., via one or more intermediate network nodes, such as base stations) a plurality of WDs to perform training of local machine learning models for federated learning, where the cloud node aggregates the local machine learning models into an aggregate model. In this case, the cloud node is the server, while the WDs are the clients. Other configurations and arrangements of “servers” and “clients” may be applied without deviating from the scope of the present disclosure. Note that although terminology from one particular wireless system, such as, for example, 3GPP LTE and/or New Radio (NR), may be used in this disclosure, this should not be seen as limiting the scope of the disclosure to only the aforementioned system. Other wireless systems, including without limitation Wide Band Code Division Multiple Access (WCDMA), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband (UMB) and Global System for Mobile Communications (GSM), may also benefit from exploiting the ideas covered within this disclosure. Note further, that functions described herein as being performed by a wireless device or a network node may be distributed over a plurality of wireless devices and/or network nodes. In other words, it is contemplated that the functions of the network node and wireless device described herein are not limited to performance by a single physical device and, in fact, can be distributed among several physical devices. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. Some embodiments provide configurations for supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. Referring now to the drawing figures, in which like elements are referred to by like reference numerals, there is shown in FIG. 1 a schematic diagram of a communication system 10, according to an embodiment, such as a 3GPP-type cellular network that may support standards such as LTE and/or NR (5G), which comprises an access network 12, such as a radio access network, and a core network 14. The access network 12 comprises a plurality of network nodes 16a, 16b, 16c (referred to collectively as network nodes 16), such as NBs, eNBs, gNBs or other types of wireless access points, each defining a corresponding coverage area 18a, 18b, 18c (referred to collectively as coverage areas 18). The communication system 10 may include a server 19, which may be a physical and/or virtual computing device, which may be located in one or more of access network 12, core network 14, etc., or may be separate from the access network 12, core network 14, etc. Each network node 16a, 16b, 16c is connectable to the core network 14 over a wired or wireless connection 20. A first wireless device (WD) 22a located in coverage area 18a is configured to wirelessly connect to, or be paged by, the corresponding network node 16a. A second WD 22b in coverage area 18b is wirelessly connectable to the corresponding network node 16b. While a plurality of WDs 22a, 22b (collectively referred to as wireless devices 22) are illustrated in this example, the disclosed embodiments are equally applicable to a situation where a sole WD is in the coverage area or where a sole WD is connecting to the corresponding network node 16. Note that although only one server 19, two WDs 22 and three network nodes 16 are shown for convenience, the communication system may include many more servers 19, WDs 22, and network nodes 16. Further, although the server 19 is illustrated as a separate device in the example of FIG. 1, in some embodiments of the present disclosure, server 19 may be implemented by a network node 16, a WD 22, or another entity of communication system 10. In describing some embodiments of the present disclosure, network nodes 16 and/or WDs 22 may be referred to as a client 23 or clients 23, collectively. It is to be understood that other types of computing devices, e.g., fixed devices, non-wireless devices, etc., may also operate as clients 23 within the context of some embodiments of the present disclosure. Also, it is contemplated that a WD 22 can be in simultaneous communication and/or configured to separately communicate with more than one network node 16 and more than one type of network node 16 (and/or server 19). For example, a WD 22 can have dual connectivity with a network node 16 that supports LTE and the same or a different network node 16 that supports NR. As an example, WD 22 can be in communication with an eNB for LTE/E-UTRAN and a gNB for NR/NG-RAN. The communication system 10 may itself be connected to a host computer 24, which may be embodied in the hardware and/or software of a standalone server, a cloud-implemented server, a distributed server or as processing resources in a server farm. The host computer 24 may be under the ownership or control of a service provider, or may be operated by the service provider or on behalf of the service provider. The connections 26, 28 between the communication system 10 and the host computer 24 may extend directly from the core network 14 to the host computer 24 or may extend via an optional intermediate network 30. The intermediate network 30 may be one of, or a combination of more than one of, a public, private or hosted network. The intermediate network 30, if any, may be a backbone network or the Internet. In some embodiments, the intermediate network 30 may comprise two or more sub-networks (not shown). In some embodiments, server 19 may be part of host computer 24, and/or may be connected to one or more network nodes 16 or WD 22s via, e.g., intermediate network 30. The communication system of FIG. 1 as a whole enables connectivity between one of the connected WDs 22a, 22b and the host computer 24. The connectivity may be described as an over-the-top (OTT) connection. The host computer 24 and the connected WDs 22a, 22b are configured to communicate data and/or signaling via the OTT connection, using the access network 12, the core network 14, any intermediate network 30 and possible further infrastructure (not shown) as intermediaries. The OTT connection may be transparent in the sense that at least some of the participating communication devices through which the OTT connection passes are unaware of routing of uplink and downlink communications. For example, a network node 16 may not or need not be informed about the past routing of an incoming downlink communication with data originating from a host computer 24 to be forwarded (e.g., handed over) to a connected WD 22a. Similarly, the network node 16 need not be aware of the future routing of an outgoing uplink communication originating from the WD 22a towards the host computer 24. In some embodiments, the OTT connection may be used to connect clients 23 (e.g., WDs 22) with server 19 (e.g., a standalone server 19, a server 19 which is part of host computer 24, etc.). A network node 16 is configured to include a NN client AI unit 32 which is configured for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system. A WD 22 is configured to include a WD client AI unit 33 which is configured for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system A server 19 is configured to include a server AI unit 34 which is configured for supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. Example implementations, in accordance with an embodiment, of the WD 22, network node 16, server 19, and host computer 24 discussed in the preceding paragraphs will now be described with reference to FIG. 2. In a communication system 10, a host computer 24 comprises hardware (HW) 38 including a communication interface 40 configured to set up and maintain a wired or wireless connection with an interface of a different communication device of the communication system 10. The host computer 24 further comprises processing circuitry 42, which may have storage and/or processing capabilities. The processing circuitry 42 may include a processor 44 and memory 46. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 42 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 44 may be configured to access (e.g., write to and/or read from) memory 46, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory). Processing circuitry 42 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by host computer 24. Processor 44 corresponds to one or more processors 44 for performing host computer 24 functions described herein. The host computer 24 includes memory 46 that is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 48 and/or the host application 50 may include instructions that, when executed by the processor 44 and/or processing circuitry 42, causes the processor 44 and/or processing circuitry 42 to perform the processes described herein with respect to host computer 24. The instructions may be software associated with the host computer 24. The software 48 may be executable by the processing circuitry 42. The software 48 includes a host application 50. The host application 50 may be operable to provide a service to a remote user, such as a WD 22 connecting via an OTT connection 52 terminating at the WD 22 and the host computer 24. In providing the service to the remote user, the host application 50 may provide user data which is transmitted using the OTT connection 52. The “user data” may be data and information described herein as implementing the described functionality. In one embodiment, the host computer 24 may be configured for providing control and functionality to a service provider and may be operated by the service provider or on behalf of the service provider. The processing circuitry 42 of the host computer 24 may enable the host computer 24 to observe, monitor, control, transmit to and/or receive from the network node 16, server 19, and or the wireless device 22. The processing circuitry 42 of the host computer 24 may include a Cloud Configuration unit 54 configured to enable the service provider to observe/monitor/control/transmit to/receive from/configure/etc. the network node 16, server 19, and/or the wireless device 22. The communication system 10 further includes a network node 16 provided in a communication system 10 and including hardware 58 enabling it to communicate with the host computer 24, the server 19, and with the WD 22. The hardware 58 may include a communication interface 60 for setting up and maintaining a wired or wireless connection with an interface of a different communication device of the communication system 10, as well as a radio interface 62 for setting up and maintaining at least a wireless connection 64 with a WD 22 located in a coverage area 18 served by the network node 16. The radio interface 62 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers. The communication interface 60 may be configured to facilitate a connection 66 to the host computer 24 and/or server 19. The connection 66 may be direct or it may pass through a core network 14 of the communication system 10 and/or through one or more intermediate networks 30 outside the communication system 10. In the embodiment shown, the hardware 58 of the network node 16 further includes processing circuitry 68. The processing circuitry 68 may include a processor 70 and a memory 72. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 68 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 70 may be configured to access (e.g., write to and/or read from) the memory 72, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory). Thus, the network node 16 further has software 74 stored internally in, for example, memory 72, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the network node 16 via an external connection. The software 74 may be executable by the processing circuitry 68. The processing circuitry 68 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by network node 16. Processor 70 corresponds to one or more processors 70 for performing network node 16 functions described herein. The memory 72 is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 74 may include instructions that, when executed by the processor 70 and/or processing circuitry 68, causes the processor 70 and/or processing circuitry 68 to perform the processes described herein with respect to network node 16. For example, processing circuitry 68 of the network node 16 may include NN client AI unit 32 configured for supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. The communication system 10 further includes the WD 22 already referred to. The WD 22 may have hardware 80 that may include a radio interface 82 configured to set up and maintain a wireless connection 64 with a network node 16 serving a coverage area 18 in which the WD 22 is currently located. The radio interface 82 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and/or one or more RF transceivers. The hardware 80 of the WD 22 further includes processing circuitry 84. The processing circuitry 84 may include a processor 86 and memory 88. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 84 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 86 may be configured to access (e.g., write to and/or read from) memory 88, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory). Thus, the WD 22 may further comprise software 90, which is stored in, for example, memory 88 at the WD 22, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the WD 22. The software 90 may be executable by the processing circuitry 84. The software 90 may include a client application 92. The client application 92 may be operable to provide a service to a human or non-human user via the WD 22, with the support of the host computer 24. In the host computer 24, an executing host application 50 may communicate with the executing client application 92 via the OTT connection 52 terminating at the WD 22 and the host computer 24. In providing the service to the user, the client application 92 may receive request data from the host application 50 and provide user data in response to the request data. The OTT connection 52 may transfer both the request data and the user data. The client application 92 may interact with the user to generate the user data that it provides. The processing circuitry 84 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by WD 22. The processor 86 corresponds to one or more processors 86 for performing WD 22 functions described herein. The WD 22 includes memory 88 that is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 90 and/or the client application 92 may include instructions that, when executed by the processor 86 and/or processing circuitry 84, causes the processor 86 and/or processing circuitry 84 to perform the processes described herein with respect to WD 22. For example, processing circuitry 84 of the WD 22 may include WD client AI unit 33 configured for supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. The communication system 10 further includes a server 19 provided in a communication system 10 and including hardware 94 enabling it to communicate with the host computer 24, with one or more network nodes 16, and with one or more WD 22. The hardware 94 may include a communication interface 96 for setting up and maintaining a wired or wireless connection with an interface of a different communication device of the communication system 10. The communication interface 96 may be configured to facilitate a connection 66 to the host computer 24 and/or one or more network nodes 16 and/or WDs 22. The connection 66 may be direct or it may pass through a core network 14 of the communication system 10 and/or through one or more intermediate networks 30 outside the communication system 10. In the embodiment shown, the hardware 94 of the server 19 further includes processing circuitry 98. The processing circuitry 98 may include a processor 100 and a memory 102. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 98 may comprise integrated circuitry for processing and/or control, e.g., one or more processors and/or processor cores and/or FPGAs (Field Programmable Gate Array) and/or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 100 may be configured to access (e.g., write to and/or read from) the memory 102, which may comprise any kind of volatile and/or nonvolatile memory, e.g., cache and/or buffer memory and/or RAM (Random Access Memory) and/or ROM (Read-Only Memory) and/or optical memory and/or EPROM (Erasable Programmable Read-Only Memory). Thus, the server 19 further has software 104 stored internally in, for example, memory 102, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the server 19 via an external connection. The software 104 may be executable by the processing circuitry 98. The processing circuitry 98 may be configured to control any of the methods and/or processes described herein and/or to cause such methods, and/or processes to be performed, e.g., by server 19. Processor 100 corresponds to one or more processors 100 for performing server 19 functions described herein. The memory 102 is configured to store data, programmatic software code and/or other information described herein. In some embodiments, the software 104 may include instructions that, when executed by the processor 100 and/or processing circuitry 98, causes the processor 100 and/or processing circuitry 98 to perform the processes described herein with respect to server 19. For example, processing circuitry 98 of the server 19 may include server AI unit 34 configured configurations supporting probabilistic client selection and resource allocation for federated learning in a wireless communication system. In some embodiments, the inner workings of the network node 16, server 19, WD 22, and host computer 24 may be as shown in FIG. 2 and independently, the surrounding network topology may be that of FIG. 1. In FIG. 2, the OTT connection 52 has been drawn abstractly to illustrate the communication between the host computer 24 and the wireless device 22 via the network node 16, without explicit reference to any intermediary devices and the precise routing of messages via these devices. Network infrastructure may determine the routing, which it may be configured to hide from the WD 22 or from the service provider operating the host computer 24, or both. While the OTT connection 52 is active, the network infrastructure may further take decisions by which it dynamically changes the routing (e.g., on the basis of load balancing consideration or reconfiguration of the network). The wireless connection 64 between the WD 22 and the network node 16 is in accordance with the teachings of the embodiments described throughout this disclosure. One or more of the various embodiments improve the performance of OTT services provided to the WD 22 using the OTT connection 52, in which the wireless connection 64 may form the last segment. More precisely, the teachings of some of these embodiments may improve the data rate, latency, and/or power consumption and thereby provide benefits such as reduced user waiting time, relaxed restriction on file size, better responsiveness, extended battery lifetime, etc. In some embodiments, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 52 between the host computer 24 (and/or server 19) and WD 22, in response to variations in the measurement results. The measurement procedure and/or the network functionality for reconfiguring the OTT connection 52 may be implemented in the software 48 of the host computer 24 or in the software 90 of the WD 22, or both. In embodiments, sensors (not shown) may be deployed in or in association with communication devices through which the OTT connection 52 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software 48, 90 may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 52 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not affect the network node 16, and it may be unknown or imperceptible to the network node 16. Some such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary WD signaling facilitating the host computer’s 24 measurements of throughput, propagation times, latency and the like. In some embodiments, the measurements may be implemented in that the software 48, 90 causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 52 while it monitors propagation times, errors, etc. Thus, in some embodiments, the host computer 24 includes processing circuitry 42 configured to provide user data and a communication interface 40 that is configured to forward the user data to a cellular network for transmission to the WD 22. In some embodiments, the cellular network also includes the network node 16 with a radio interface 62. In some embodiments, the network node 16 is configured to, and/or the network node’s 16 processing circuitry 68 is configured to perform the functions and/or methods described herein for preparing/initiating/maintaining/supporting/ending a transmission to the WD 22, and/or preparing/terminating/maintaining/ supporting/ending in receipt of a transmission from the WD 22. In some embodiments, the host computer 24 includes processing circuitry 42 and a communication interface 40 that is configured to a communication interface 40 configured to receive user data originating from a transmission from a WD 22 to a network node 16. In some embodiments, the WD 22 is configured to, and/or comprises a radio interface 82 and/or processing circuitry 84 configured to perform the functions and/or methods described herein for preparing/initiating/maintaining/supporting/ending a transmission to the network node 16, and/or preparing/terminating/maintaining/ supporting/ending in receipt of a transmission from the network node 16. Although FIGS. 1 and 2 show various “units” such as NN client AI unit 32, WD client AI unit, and server AI unit 34 as being within a respective processor, it is contemplated that these units may be implemented such that a portion of the unit is stored in a corresponding memory within the processing circuitry. In other words, the units may be implemented in hardware or in a combination of hardware and software within the processing circuitry. FIG. 3 is a flowchart illustrating an example method implemented in a communication system, such as, for example, the communication system of FIGS. 1 and 2, in accordance with one embodiment. The communication system may include a host computer 24, a network node 16 and a WD 22, which may be those described with reference to FIG. 2. In a first step of the method, the host computer 24 provides user data (Block S100). In an optional substep of the first step, the host computer 24 provides the user data by executing a host application, such as, for example, the host application 50 (Block S102). In a second step, the host computer 24 initiates a transmission carrying the user data to the WD 22 (Block S104). In an optional third step, the network node 16 transmits to the WD 22 the user data which was carried in the transmission that the host computer 24 initiated, in accordance with the teachings of the embodiments described throughout this disclosure (Block S106). In an optional fourth step, the WD 22 executes a client application, such as, for example, the client application 92, associated with the host application 50 executed by the host computer 24 (Block S108). FIG. 4 is a flowchart illustrating an example method implemented in a communication system, such as, for example, the communication system of FIG. 1, in accordance with one embodiment. The communication system may include a host computer 24, a network node 16 and a WD 22, which may be those described with reference to FIGS. 1 and 2. In a first step of the method, the host computer 24 provides user data (Block S110). In an optional substep (not shown) the host computer 24 provides the user data by executing a host application, such as, for example, the host application 50. In a second step, the host computer 24 initiates a transmission carrying the user data to the WD 22 (Block S112). The transmission may pass via the network node 16, in accordance with the teachings of the embodiments described throughout this disclosure. In an optional third step, the WD 22 receives the user data carried in the transmission (Block S114). FIG. 5 is a flowchart illustrating an example method implemented in a communication system, such as, for example, the communication system of FIG. 1, in accordance with one embodiment. The communication system may include a host computer 24, a network node 16 and a WD 22, which may be those described with reference to FIGS. 1 and 2. In an optional first step of the method, the WD 22 receives input data provided by the host computer 24 (Block S116). In an optional substep of the first step, the WD 22 executes the client application 92, which provides the user data in reaction to the received input data provided by the host computer 24 (Block S118). Additionally or alternatively, in an optional second step, the WD 22 provides user data (Block S120). In an optional substep of the second step, the WD provides the user data by executing a client application, such as, for example, client application 92 (Block S122). In providing the user data, the executed client application 92 may further consider user input received from the user. Regardless of the specific manner in which the user data was provided, the WD 22 may initiate, in an optional third substep, transmission of the user data to the host computer 24 (Block S124). In a fourth step of the method, the host computer 24 receives the user data transmitted from the WD 22, in accordance with the teachings of the embodiments described throughout this disclosure (Block S126). FIG. 6 is a flowchart illustrating an example method implemented in a communication system, such as, for example, the communication system of FIG. 1, in accordance with one embodiment. The communication system may include a host computer 24, a network node 16 and a WD 22, which may be those described with reference to FIGS. 1 and 2. In an optional first step of the method, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 16 receives user data from the WD 22 (Block S128). In an optional second step, the network node 16 initiates transmission of the received user data to the host computer 24 (Block S130). In a third step, the host computer 24 receives the user data carried in the transmission initiated by the network node 16 (Block S132). FIG. 7 is a flowchart of an example process in a sever 19 according to some embodiments of the present disclosure. One or more blocks described herein may be performed by one or more elements of server 19 such as by one or more of processing circuitry 98 (including the server AI unit 34), processor 100, and/or communication interface 96. Server 19 is configured to initialize (Block S134) an aggregate machine learning model for federated learning. Server 19 is configured to perform (Block S136) a federated learning on the aggregate machine learning model with the plurality of clients 23 for a plurality of rounds, where a round of the federated learning includes determining an optimal subset of the plurality of clients 23, causing transmission, to at least the optimal subset of the plurality of clients 23, of a first indication of a current state of the aggregate machine learning model, receiving, from the optimal subset of the plurality of clients 23, a plurality of second indications corresponding to local machine learning models associated with the optimal subset of the plurality of clients 23, and updating the aggregate machine learning model by aggregating the local machine learning models. In some embodiments, the server 19 is further configured to transmit and/or cause transmission of an initial configuration indication to the plurality of clients 23, where the initial configuration indication includes at least one of a local learning rate, and a number of local steps. In some embodiments, the server 19 is further configured to initialize the aggregate machine learning model by initializing at least one virtual queue, and perform the federated learning by updating the at least one virtual queue based on the local machine learning models. In some embodiments, the server 19 is further configured to, for each round of the federated learning, transmit and/or cause transmission of, to all of the plurality of clients 23, the first indication of the current state of the aggregate machine learning model, for each round of the federated learning, receive, responsive to the first indication, a plurality of third indications of a plurality of stochastic gradients associated with the plurality of clients 23, and for each round of the federated learning, the determining of the corresponding optimal subset of the plurality of clients 23 is based on the plurality of stochastic gradients. In some embodiments, the determining of the optimal subset of the plurality of clients 23 is based on a convex optimization of at least one of radio channel resources associated with the plurality of clients 23, processing resources associated with the plurality of clients 23, and power resources associated with the plurality of clients 23. In some embodiments, the server 19 is further configured to receive radio measurement data associated with at least one client 23 of the plurality of clients 23, and predict a location of the at least one client 23 based on the radio measurement data using the aggregate machine learning model. FIG. 8 is a flowchart of an example process in a client 23 (e.g., a network node 16 and/or a WD 22) according to some embodiments of the present disclosure. One or more blocks described herein may be performed by one or more elements of wireless device 22 such as by one or more of processing circuitry 84 (including the WD client AI unit 33), processor 86, radio interface 82, and/or by one or more elements of network node 16, such as communication interface 60, radio interface 62, processing circuitry 68, memory 72, processor 70, NN client AI unit 32, etc. Client 23 is configured to at least one of receive, store, and determine (Block S138) a dataset for training a local machine learning model. Client 23 is configured to perform (Block S140) a federated learning for an aggregate machine learning model with the plurality of additional clients 23 for a plurality of rounds, where a round of the federated learning includes, receiving, from the server 19, a first indication of a current state of the aggregate machine learning model, training, in parallel with an optimal subset of the plurality of additional clients 23, a local machine learning model based on the current state of the aggregate machine learning model and the dataset, and transmitting and/or causing transmission of, to the server 19, a second indication corresponding to the local machine learning model. In some embodiments, the client 23 is further configured to receive an initial configuration indication from the server 19, where the initial configuration indication includes at least one of a local learning rate, and a number of local steps, and the training of the local machine learning model is based on the local learning rate and the number of local steps. In some embodiments, the client 23 is further configured to, for each round of the federated learning, receive the first indication of the current state of the aggregate machine learning model, for each round of the federated learning, determine a stochastic gradient based on the local machine learning model, and for each round of the federated learning, cause transmission, responsive to the first indication, of a third indication of the stochastic gradient, where the optimal subset of the plurality of clients 23 is determined based on the stochastic gradient. In some embodiments, the optimal subset of the plurality of clients 23 is determined based on a convex optimization of at least one of radio channel resources associated with the plurality of clients 23, processing resources associated with the plurality of clients 23, and power resources associated with the plurality of clients 23. In some embodiments, the client 23 is further configured to cause transmission, to the server 19, of radio measurement data associated with at least one client 23 of the plurality of clients 23, and receive a location prediction based on the radio measurement data and the aggregate machine learning model. Having described the general process flow of arrangements of the disclosure and having provided examples of hardware and software arrangements for implementing the processes and functions of the disclosure, the sections below provide details and examples of arrangements for supporting configurations for probabilistic client selection and resource allocation for federated learning in a wireless communication system. Federated Learning (FL) with Probabilistic Sampling Embodiments of the present disclosure may provide configurations for minimizing the global loss ^^^^ of FL, such as by performing computations (e.g., in any one or more of processing circuitry 68, processing circuitry 84, processing circuitry 98, NN client AI unit 32, WD client AI unit 33, and/or server AI unit 34) according to the following equation: ^ 23 (e.g., network nodes 16, WDs 22, etc., participating in the federated learning), and ^_^ is the loss incurred by sample ^. Let ^ be the total number of clients, ^_^ be the number of samples on client 23 ^ that are sampled from ^_^, and ^_^ be the index set of samples on client 23 ^. In some embodiments, ^_^^^^ = ^{^} ^_^ ∑_{^ ∈ ^^} ^_^^^^ may define refer to the local loss of data on client 23 ^ (e.g., a network node 16n, WD 22n, etc.). In some embodiments, it may be assumed that all ^_^ have the same domain as of ^. Let p_' = D_'/D be the ratio between number of local data samples on client 23 ^ and total number of data samples among all clients 23. It may be assumed, for example, that p_' > 0, ∀n ∈ -N^/, i.e., each client 23 n has at least one sample. Equivalently, the global loss f^{^}w^{^} may be, in some embodiments, a weighted sum of the local losses f_' ^{^}w^{^}, so (Eq. Eq. 1) may be rewritten as: 3 In some framework may utilize a uniformly random selection of clients, or may utilize a probabilistic client sampling, as described herein, which may provide one or more benefits over existing systems, such as higher efficiency, accuracy, etc. In some embodiments, the sampling probabilities for all clients of all global rounds may be provided and/or assumed. Some embodiments may include an aggregation of local models, wherein each aggregation results in an unbiased estimator of the weighted sum of all local results, e.g., based on (Eq. Eq. 2). In describing some embodiments of the present disclosure, -5/ may be used as an abbreviation of the set {0, 1, … , T}. FIG. 9 is a flowchart illustrating a first example process flow and algorithm, (referred to herein as “Algorithm 1”), as implemented in a server 19, clients 23, etc., according to some embodiments of the present disclosure. In the example of FIG. 9, the “FedAvg” function may be performed according to a probabilistic client sampling approach. It may be assumed that the server 19 knows the exact (or nearly exact) number of data samples on each client (e.g., each of the plurality of network nodes 16, WDs 22, etc., participating in the federated learning). Referring to FIG. 9, the server 19 initializes the global model (Step S150), including setting initial variables (e.g., initializing timing variable(s) such as t = 0, initializing the weight variable(s), calculating a data ratio, etc.). The server 19 may broadcast parameters for the local learning procedures to be performed at clients 23 (e.g., network nodes 16 and/or WDs 22), such as local learning rate information, number of local steps, etc., to each of the clients 23 participating in the federated learning). For each round : (Step S154), (a) the server 19 realizes the set of selected clients 23 by running an independent Bernoulli trial for each client 23 ^ with probability q^' _< (Step S156), and sends the current model parameters to each selected client 23 n; (b) each selected client 23 n then runs (Step S158) SGD or other gradient based methods to update the model using its own data, and sends back its updated model to the server 19 after performing its local computation (Step S158); (c) after the updated models of all selected clients 23 n arrive, the server 19 aggregates them to update the global model (Step S160), such as according to Eq. 3, defined below. A counter may be incremented after each loop iteration (Step S162). The loop may end after 5 rounds or some predefined requirements are satisfied (Step S164). For example, the loop may end after 5 rounds or some other predefined requirements like accuracy improvement drops below a threshold are satisfied. The variable w_< may represent the parameters of the global model after :-th aggregation, and w_< ^' ,₌ may represent the parameters of the local model after :-th aggregation and ^-th local SGD ^{step on client ^. Let > be the number of local steps, 5 be the number of global rounds,} ^{? be the local learning rate, a'} _< ^{be the indicator function representing whether client 23} _{^ is selected at global round :, and g' is a (stochastic) gradient of local loss function} _f'. In some embodiments, the aggregation (e.g., as described with respect to Algorithm 1) may be an unbiased estimator of the model, e.g., as if all the clients 23 are participating in each round. L^{emma 1. w<B^ is an unbiased estimator of ∑D '} ^{'%^ p'w<,C , i.e., E-w<B^/ =} _∑D _{'%^ p'w'} _{<,C , where} 3 In some embodiments, a convergence analysis may be performed for the Federated Averaging (FedAvg) function with probabilistic client sampling algorithm, e.g., with non-convex and smooth loss functions and taking into account statistical heterogeneity. For example let dom^^^ be the domain of any function ^. In some embodiments, one or more of the following assumptions, Assumption 1, Assumption 2, and Assumption 3, may apply: Assumption 1 (Smoothness). Each f_' ^{^}x^{^} is O-smooth: − ≤ − ∀R, y ∈ dom^{^}^_^ ^{^}. (Eq. 4) Assumption 1 may imply a global loss ^^R^ is also O-smooth, which can be proven by Jensen's inequality, for example. Assumption 2 (Unbiased local stochastic gradient). The stochastic gradient g_' is an unbiased estimator of the true gradient for any local loss ^_^. E-X_^^S^/ = ∇^_^^S^, ∀S ∈ dom^^_^^. (Eq. 5) This assumption may be a standard assumption used for the analysis of many stochastic gradient methods. For Stochastic Gradient Descent (SGD) or mini-batch SGD, for example, if the samples are sampled uniformly at random from the data, the stochastic gradient may be an unbiased estimator of the true gradient with respect to the loss function, e.g., as if the servers 19 and/or clients 23 were using all of the data, rather than a sample. Assumption 3 (Bounded stochastic gradients): There exist G_' > 0 such that E [P^|g_' ^{^}y^{^|}P^\ ] ≤ G^\ _' , (Eq. 6) holds for all y ∈ dom^{^}f_' ^{^} and n ∈ -N^/, where g_' is a stochastic gradient of f_'. Some embodiments may use potentially different upper bounds for the expectation of squared norms of stochastic gradients on each client 23, e.g., as one way to capture the statistical heterogeneity. In other words, in some embodiments, there may be different G_' for each client 23 ^ instead of a universal ^ for all clients 23. Theorem 1. Suppose Assumptions 1-3 holds, then it may be shown that: _^{`^} where of (Eq. 1). It may be observed that only the third term of the upper bound related to I: _`^ 3 K`^ Based on this the convergence bound in various optimization formulations and strategies of the embodiments described herein. For case one, the upper bound of the norm of the stochastic gradient may be applied as in Assumption 3 for (Eq. Eq. 8): _`^ 3 O^?> _$ ^{2^ \} 5_$ _{I^ ^^} . (Eq. 9) F_%a ^%^ _F For case two, the actual norm of the stochastic gradient at global round : may be used, denoted by ^_F ^{^}, i.e., ^_F ^{^} = ∑^K ^_% ^` a^{^} || X_^J^_F ^{^} ,_^ M||^\, and the expression becomes: _`^ 3 In describing the general formula utilizing ^_F ^{^}, but it is to be understood that embodiments may also be applicable for the case ^_^ is independent of :, e.g., as in the formula of case ^{one. If the only constraints imposed on q'} ^{< are the probability constraints, i.e., 0 ≤} q^' _< ≤ 1, ∀n ∈ -N^/, then (Eq. Eq. 10) may be minimized when all I_F ^{^} are 1 since all _p'^G' _{< ^\ are positive as well as O, ?, >, and 5. In some cases, it may not be possible} to control the expected number of selected clients 23 at each round, e.g., as all clients 23 are chosen for all rounds, i.e., full participation for all rounds. Instead, constraints may be imposed by server 19 on the expected number of select clients 23 while minimizing the objective, which may provide closed-form solutions to the equations described herein. Joint Probabilistic Client Sampling and Resource Allocation Algorithms In some embodiments, the online resource allocation formulas may be computed in conjunction with (Eq. Eq. 10). For example, assuming there are g different types of resources h_^, … , h_i, the j-th resource allocated to clients at time slot : may be denoted as k_l,F = {u^{^} n_,< , … , u^D n_,< } where u^' n_,< is the j-th resource allocated to client n ∈ -N^/ at time :. The cost function for allocating resource j to client ^ at time : may be defined as ϕ^' n_,< : h_n → R_B. Hence, the cost incurred by allocating s_l ^{^} ,_F may be written as ϕ^' n_{,< n,<} expectation of the weighted sum of these g resources allocated to different clients at time : as part of the optimization objective may be computed by server 19 or clients 23 as: 3 u 3 M (Eq. 11) where λ_n > q^' _< is due to the client sampling. In some embodiments, long-term time-average constraints and per-slot short- term constraints may be utilized by server 19 in computing a solution to a stochastic optimization problem, as described herein. For example, for each resource j, the quantity whose expected time-average we would like to constraint may be defined according to c_n ^' ,_< : h_n → R_B for each client 23 ^ at time :. The constraints for actions may The actions may include, e.g., the sampling probabilities at all time slots. It may be assumed that each I_F ^{^} is between 0 and 1 inclusively and the summation of I_F ^{^} over ^ ∈ -^/ for each time : is upper bounded by m ∈ ^0, N/, i.e., the expected number of selected clients 23 at each time slot is no greater than y. The stochastic optimization problem may be formulated (e.g., for computation at server 19 and/or clients 23) as: _^{`^} the time average of (Eq. Eq. 11). Following the theory and analysis of Lyapunov optimization, the original stochastic optimization problems on time averages may be transformed into a problem of queue stabilities. For example, let: S_l ^{^}^:^ = I_F ^{^}^_l ^{^} Js_l ^{^} M. (Eq. 17) Virtual queues may rules may include: ^_l ^{^^}: + 1^{^} = max{ ^_l ^{^^}:^{^} + S_l ^{^^}:^{^}, 0} (Eq. 18) ^^:^ may represent all the queue backlogs at time :, e.g., as if they may be stacked into one vector. The Lyapunov function which measures the stabilities of queues may be defined as: u ³ The Lyapunov Δ^^^:^^ ≔ E {O^^^: + 1^^ − O^^^:^^ | O^^^:^^}. (Eq. 20) The drift-plus-penalty expression may be defined as: Δ^^^t^^ + VE{S_a ^{^}:^{^}|^^t^}, (Eq. 21) where ^ ∈ ^_B plays the role of deciding the trade-off between the drift of the Lyapunov function and minimizing the objective functions: a higher value of ^ indicates more weight on the objective functions and a lower value of ^ means heavier importance we want to impose on the drift term. From the theory of Lyapunov optimization, the upper bound of the drift-plus-penalty expression may be derived, e.g.: Δ^{^^^:^^ +} where ^_a provided for opportunistically minimizing the upper bound at server 19, which may define the following per-slot problem: u ³ defined in Equations (Eq. Eq. 23)-(Eq. Eq. 26), may be non-convex in general. Also, it may not be feasible to directly separate it into ^ subproblems, one for each client 23 ^, as there may be a constraint on the sum of sampling probabilities in (Eq. Eq. 25), for example, which coalesces input from different clients 23. It may further be assumed that all ϕ^' n_,< and c_n ^' ,_< are convex functions. Several concrete examples for these herein. Based on the convexity, the following observations may apply: F^{or any feasible {I^} ^{F }3} ^{^%^ , the objective and constraints are convex in} {_{u }i , and the optimal values of {k }u are unrelated t ^ 3} _{n,< n%^ l,F l%^ o {IF }^%^ ;} If {k }^u are fixed, the constraints in {I^{^}}³ ^_%^ detailed solution of the per-slot optimization problem is described as follows. If {I_F ^{^}}³ ^_%^ are fixed, the optimization problem of {k_l,F}^u l_%^ becomes: u ³ It may resource j as there is no cross-term that relates to more than one resource. Hence, for each resource j, the following optimization problem may apply: 3 3 individual optimization of u^' n_,< as there is no cross-term with respect to more than one client 23. It may also be observed that the I_F ^{^} terms in the objective can be eliminated. minimize ^{^} ^^_l^^{^} Js^{^} M + ^_l ^{^}^:^^^{^} Js^{^} M I^{t is known that ^, ^l, and ^^} ^{l^:^ are non-negative. From the convexity of} ^_^ _{l,F and ^^} _{l,F , it may also be known that the objective is convex as it is a non-negative} weighted sum of convex functions. Hence, an optimization problem of one variable with a convex objective and a box constraint may be utilized by server 19, in some embodiments. Standard convex optimization solvers may be utilized to solve the problem, or the optimization problem may have a closed-form solution, if it exists. If {k_l,F}^u l_%^ are fixed, the optimization problem becomes: u ³ minimize_{{| ^{^} _}^ _{^^^ ,} ^S_a ^{^}:^{^} + $ $ ^_l ^{^^}:^{^}S_l ^{^^}:^{^} (Eq. 34) l_{%^ ^%^} subject to ^∑3 ^_%^ I_F ^{^} ≤ y, (Eq. 35) 0 ≤ I_F ^{^} ≤ 1, ∀^ ∈ -^^/ (Eq. 36) The objective is: 3 u 3 This is a and the constraints are affine in {I_F ^{^}}³ ^_%^ . A optimization solver may be used by server 19 to achieve the optimal solution based thereon. H^{ence the subproblems may be solved to get {k u} ^{l,F}l%^ , then, by solving for} _{I^ _{F }3} _{^%^ conditioned on the values of {kl,F}u} _{l%^ , an optimal solution may be obtained} by server 19. For example, the following may apply: ^^{`^} where y^opt guaranteed in some cases that all time average constraints are satisfied when t → ∞. FIG. 10 illustrates a Probabilistic Client Selection and Resource (PCSRA) performed by server 19 and/or clients 23, according to some embodiments of the present disclosure, also referred to herein as “Algorithm 2”. In Step 172, the server 19 sets t = 0, estimates the gradient, initializes various parameters and virtual queues, and calculates a data ratio. In Step 174, the server 19 broadcasts to one or more clients 23 which are participating in the federated learning (e.g., network nodes 16, WDs 22, etc.), a local learning rate, number of local steps, and any other parameters which are configured to be broadcast. In Step 176, it is determined whether the number of steps has exceeded the maximum T, and if so, the loop exits at Step S178. Otherwise, the server 19 calculates the solution(s) to the convex optimization problem, e.g., according to Equations (Eq. 23) – (Eq. 26) (Step S180), and the server 19 selects clients 23 according to one or more techniques described herein, and broadcasts the current model parameters (e.g., weight vector(s)) to each selected client 23 (Step S182). Each client 23 n runs local computations (e.g., in parallel), such as using processing circuitry 84, and transmits the resulting local model back to the server 19 (Step S184). The server 19 is configured to aggregate the received local models, e.g., according to (Eq. 3) (Step S186), and the server 19 updates the virtual queues, e.g., via (Eq. 18) (Step S188). The step counter is incremented (Step S190), and the loop reiterates accordingly (Step S192). FIG. 11 illustrates a Dynamic Probabilistic Client Selection and Resource (Dynamic-PCSRA) performed by server 19 and/or clients 23, according to some embodiments of the present disclosure, also referred to herein as “Algorithm 3”. In Step S194, the server 19 sets the counter t to zero, initializes the weight vector(s) and any other parameters/variables of the AI model, initializes the virtual queues, and calculates a data ratio. In Step S196, the server 19 broadcasts to one or more clients 23 which are participating in the federated learning (e.g., network nodes 16, WDs 22, etc.), a local learning rate, number of local steps, and any other parameters which are configured to be broadcast. In Step 198, it is determined whether the number of steps has exceeded the maximum T, and if so, the loop exits at Step S200. Otherwise, the server 19 broadcasts the current model and any other parameters it is configured to broadcast to each client 23 n which is participating in the federated learning (Step S202). Each client 23 n runs local computation(s), e.g., in parallel, and transmits one or more parameters, such as ^_F ^{^}, back to the server 19 (S204). The server 19 performs one or more convex optimization calculations/subproblems, e.g., via Equations (Eq. 23) through (Eq. 26) (Step S206). The server 19 selects clients 23, e.g., via {I_F ^{^}}³ ^_%^ , and broadcasts {k_l,F}^u l_%^ to each selected client 23 n (Step S208). Each client 23 n resulting local model back to the server 19 (Step S210). The server 19 aggregates the local models, e.g., via (Eq. 3) (Step S212). The server 19 updates the virtual queues, e.g., via (Eq. 18) (Step S214). The counter is incremented (Step S216), and the loop reiterates (Step S218). Integrating the sampling probabilities from the solution of the per-slot optimization problem (Eq. Eq. 23)-(Eq. Eq. 26) into Algorithm 1, at least two algorithms/strategies may be obtained, e.g., according to the examples of FIG. 10 and FIG. 11. In some embodiments of PCSRA, {G_'}^D '_%^ is only estimated once before training by running one epoch of data on each client 23, while in some embodiments of Dynamic-PCSRA, {G_< ^'}^D '_%^ is calculated at each global round :. In some embodiments, after the client selection of each round, the server 19 sends not only the current global model but also the calculated resource allocation to the selected clients 23. Also, in some embodiments, such as the example of FIG. 11 (Algorithm 3/Dynamic-PCSRA), before the actual selection of clients 23 at each round, all clients 23 are configured to perform their local computations to determine the actual norm of stochastic gradients of this round, and have an extra round of transmitting only the norms. It may be assumed that the time of this extra round of communication is negligible, as it is only caused by transmitting g float numbers of resource allocation between the server 19 and each client 23 rather than all the parameters of a large model, whose size is of many orders of magnitude greater than the sizes of resources. Since the time of the extra round of communication is almost infinitesimal, it may be assumed in some embodiments that the cost functions ^_l ^{^} ,_F and constraints functions ^^{^} do l_,F not vary before and after the extra round. The general framework described herein may be applied to many real-world applications, for example, such as power allocation, by viewing power as resources and defining ^_l ^{^} ,_F as the communication time by the Shannon's bound for client ^ at time : in the power given all other parameters and defining ^_l ^{^} ,_F as the power minus the desired long-term power constraints. As another example, CPU frequency allocations may be performed by defining ^_l ^{^} ,_F as the computation time which is proportional to the reciprocal of the and defining ^_l ^{^} ,_F as the energy consumption, which is a quadratic function of the frequency, minus the desired long-term energy constraints. In the next section, an example application for power allocation, according to some embodiments of the present disclosure, is provided. Example: Joint Learning and Power allocation An example application according to some embodiments of the present disclosure for joint optimization of learning and power allocation is provided, where there is only one resource of power: g = 1 and u^' n_,< = P_< ^'. All ^_l ^{^} ,_F and ^_l ^{^} ,_F are convex functions. System Model The theoretical channel capacity may be provided according to, e.g., an upper bound by Shannon's Law, as the possible maximum transmission rate, to model the uplink communication from clients 23 to the server 19. Also, for the downlink communication time, it may be assumed that the server 19 can broadcast the aggregated global model to all the clients 23. The transmission rate r_< ^' of client ^ for the uplink communication at global round ^ may be defined as: ℎ^{^^^} ^{F F} where ^ is the channel gain of client 23 ^ at global round :, and P_< ^' is the allocated transmission power of client 23 ^ at global round :. In the wireless setting, each ℎ_F ^{^} follows exponential distribution as it may be assumed that the uplink channel of each client experiences Rayleigh fading but for the following strategies described herein with ^{respect to some embodiments, it may not be necessary to know the distribution of any} _ℎ^ _{F .} The communication time of client 23 ^ at global round : may be defined as: T_c ^' _omm,< = ^La r_' , (Eq. 40) < where L_a is the size of the model in bits. In some embodiments, it may be possible to ignore the computation time of both the server 19 and clients 23, e.g., in the optimization process. The total time for global round : may be computed by server 19 as follows, for example: 5_total,F ^{= 5} _c ^{^} _omm,F where the ine indicator functions take values of 0 or 1. An objective of the stochastic optimization includes a trade-off between the convergence bound and the round time. Defining the objective: 3 ) The stochastic optimization problem may be formulated for computation at server 19 as: _^{`^} An objective of the optimization problem which is shown in (Eq. Eq. 48) is the time average of (Eq. Eq. 47). The constraints in (Eq. Eq. 49) and (Eq. Eq. 50) are the constraints on the time average and maximum transmission power respectively. To be more specific, P_< ^' may be defined as the transmission power allocated to client 23 ^ at round :, P^¬ _' is the upper bound for the time average of the transmission power of each client 23 ^, and P_max is the maximum transmission power any client 23 can be allocated at any round. The time average in (Eq. Eq. 49) is not just the time averages of P_< ^' but those of P_< ^'q^' _< as well, in at least some cases, because the probability of picking client 23 ^ at round : is I_F ^{^} , e.g., only counting the transmission power allocated in expectation. The constraints in (Eq. Eq. 51) and (Eq. Eq. 52) may be considered constraints on control actions in some cases, wherein the sampling probabilities for each client 23 is sampled with a probability between 0 and 1 ^{(excluding 0 but including 1), and the expected number of clients 23 is no larger than} _{y, which is a positive number no greater than ^.} Following this framework, it is possible to derive the following optimization for example subproblems of power allocation, e.g., as closed-form solutions. Let S_^^_:^ _{= ^F} ^{^} _IF ^{^} _{− ^} ^¬ _{^ .} (Eq. 53) Define virtual queues: ^_^ ^{^} _{: + 1} ^{^} _{= max{ ^^} ^{^} _: ^{^} _{+ S^} ^{^} _: ^{^} _{, 0}.} ^{(Eq. 54)} The Lyapunov function which measures the stabilities of queues is defined as: 3 . ^{(Eq. 55)} The Lyapunov drift Δ^^^:^^ ≔ E {O^^^: + 1^^ − O^^^:^^ | O^^^:^^}. (Eq. 56) The drift-plus-penalty expression is defined as: Δ^^^t^^ + VE{S_a^:^|^^t^}, (Eq. 57) where ^ ∈ ^_B plays the role of deciding the trade-off between the drift of the Lyapunov function and minimizing the objective functions: a higher value of ^ indicates more weight on the objective functions and a lower value of ^ means heavier importance to impose on the drift term. From the theory of Lyapunov optimization, the upper bound of the drift-plus-penalty expression may be computed by server 19 as: Δ^{^^^:^^ +} bound, which defines the following per-slot problem: 3 A for computation at server 19. If {I_F ^{^}}³ ^_%^ are fixed, the optimization problem becomes: 3 ^{subject to 0 ≤ ^} _F ^{^ ≤ ^} _max ^{, ∀^ ∈ -^/} _{(Eq. 65)} The problem can be solved (e.g., computed by server 19 and/or clients 23) by each individual optimization of ^_F ^{^} and, by observing that the I_F ^{^} terms in the objective can be eliminated: O^a ) This may be defined as an optimization problem of one variable with a convex objective and box constraints. The minimum may be achieved at P^∗ of the unconstrained problem if P^∗ ∈ -0, P_max/, otherwise it is achieved at P_max. To find the closed-form solution of the subproblem: ^^{a ^} . If {^_F ^{^}}³ ^_%^ are fixed, the optimization problem becomes: 3 This may be computed (e.g., by server 19 and/or clients 23) as a convex optimization problem for {I_F ^{^}}³ ^_%^ as the objective is convex and the constraints are affine in {I_F ^{^}}³ ^_%^ . It may to use one or more standard convex optimization solvers to achieve an optimal solution. Hence, by solving the subproblems to get {^_F ^{^}}³ ^_%^ , and solving for {I_F ^{^}}³ ^_%^ conditioned on the values of {^_F ^{^}}³ ^_%^ , embodiments of the present support configurations for federated learning in a wireless communication system 10. In some cases, the same algorithm guarantee may be as given in (Eq. Eq. 38). Integrating the sampling probabilities from the solution of the per-slot optimization problem of (Eq. Eq. 59)-(Eq. Eq. 62) into Algorithm 1, described above (e.g., as in the example of FIG. 9), there are at least two specific versions or embodiments which include the algorithms of probabilistic client sampling and power allocation of Algorithm 2 (PCSRA) and of Algorithm 3 (Dynamic-PCSRA), where the general resources may include the allocated power. After the client 23 selection of each round, the server 19 may be configured to send not only the current global model, but also the calculated power allocation to the selected clients 23, in some embodiments. This may not contribute substantially to the communication overhead, in some embodiments, and downlink communication time may not be considered in some embodiments as part of the assumptions and/or optimization computations. In the example of Algorithm 3 (Dynamic-PCSRA), before the actual selection of clients 23 at each round, all clients 23 need to do their local computation to determine the actual norm of stochastic gradients of this round. It may be assumed that the time of this extra round of communication is negligible, in certain embodiments, as it is only caused by transmitting a float number of each client rather than all the parameters of a large model. Since the time of the extra round of communication is almost infinitesimal, in some cases, it may be assumed that the channel conditions do not vary before and after the extra communication round. Beyond communication, potentially extra computation is induced in the dynamic case as every client 23 may be configured to perform the computation. Experiments were conducted to simulate aspects of one or more embodiment of the present disclosure. For example, some simulated experiments for embodiments of the present disclosure used JAX and FedJAX for the learning framework and CVXPY for the optimization solvers. Experiments were run on classification tasks, including of images on MNIST dataset. Two data distributions were simulated: IID and non-IID. In the IID case, each client 23 was provided equal sizes of data which are drawn from the same distributions. In the non-IID case, each client 23 only contains one class, and the size of samples range from 50 to 500 (exclusive) with a step size of 50. The model in the simulated example included a fully-connected neural network with two hidden layers of 300 and 100 neurons each, and including 266610 trainable parameters, which are of size 8531520 using the default single precision float number representation. The optimizer used was SGD with learning rate 0.01 and mini-batch of size 2. A server-client architecture of N = 10 clients and the channel condition from each client to the server follows Rayleigh fading was configured, resulting in all h^' _< following an exponential distribution. Example simulation parameters are shown in Table I: Bas elines of both separate optimizations and joint optimizations were utilized, including three baselines of separate optimizations. For client sampling, the common constraints were defined for the example as: 3 Based on Sampling uniformly at random: m_inimize ^{^} _. Sampling based 3 (Eq. 75) Sampling based 3 ^' (Eq. 76) The first D, client has the same sampling probability. It corresponds to a “vanilla” version of FedAvg. The second baseline denoted as “size” adds the effects of different data ratios where, in the solution, clients 23 with larger number samples will have no smaller sampling probability than those with a smaller number of samples. The third baseline denoted as “gradient” further places the different upper bounds of the norms of the gradients into consideration. For resource allocation of the three separated optimization baselines, the long-term constraints were used as the allocations. For the joint optimizations, the problem is formulated as (Eq. Eq. 12)-(Eq. Eq. 16). The strategy based on it is referred to herein as “Dynamic-PCSRA”. The first part of the objective is substituted by each of the objectives of the three baselines in the separate optimizations, and naming the strategies based on those optimization problems Lyapunov-uniform, Lyapunov-size, and PCSRA respectively, when there ^{are no constraints on the expected number of selected clients for training, i.e., m =} _{N, it is not necessary to consider the data ratio of each client 23, and it may be} assumed that an identical bound for the norms of the gradients may apply, and the Lyapunov-uniform may be utilized. FIG. 12 is a series of graphs (a)-(d) which illustrate four example simulation outputs for IID data, according to embodiments of the present disclosure. In this example, accuracy/loss are shown with different strategies on IID data for y = 5. Graph (a) of FIG. 12 illustrates an example accuracy output of the simulation. Graph (b) of FIG. 12 illustrates an example loss output of the simulation. Graph (c) of FIG. 12 illustrates an example Lyapunov accuracy of the simulation. Graph (d) of FIG. 12 illustrates an example Lyapunov loss of the simulation. FIG. 13 is a series of graphs (a)-(d) which illustrate four example simulation outputs for IID data, according to embodiments of the present disclosure. In this example, accuracy/loss are shown with different strategies on IID data for y = 10. Graph (a) of FIG. 13 illustrates an example accuracy output of the simulation. Graph (b) of FIG. 13 illustrates an example loss output of the simulation. Graph (c) of FIG. 13 illustrates an example Lyapunov accuracy of the simulation. Graph (d) of FIG. 13 illustrates an example Lyapunov loss of the simulation. The graphs of FIG. 12 and FIG. 13 plot the accuracy and loss with respect to cumulative time for y = 5 and y = 10, for the simulations corresponding to embodiments of the present disclosure. In each of the graphs of FIG. 12 and FIG. 13, the first row (Graphs (a) and (b) of each of FIG. 12 and FIG. 13) is for the comparison among PCSRA and three baselines of separate optimization, while the second row (Graphs (c) and (d) of each of FIG. 12 and FIG. 13) is for the comparison among joint optimizations. It may be observed that PCSRA outperforms all three baselines of separate optimization, and Dynamic-PCSRA outperforms all other joint optimization strategies, with a larger gain observed with a larger y. For example, when y = 10, the three baselines of separate optimization collapse as the optimal solution is choosing each client 23 with probability one at all rounds (using the same random seed for the channel condition of each round, so the network states are exactly the same). Non-IID case FIG. 14 is a series of graphs (a)-(d) which illustrate four example simulation outputs for non-IID data, according to embodiments of the present disclosure. In this ^{example, accuracy/loss are shown with different strategies on non-IID data for y =} _{5. Graph (a) of FIG. 14 illustrates an example accuracy output of the simulation.} Graph (b) of FIG. 14 illustrates an example loss output of the simulation. Graph (c) of FIG. 14 illustrates an example Lyapunov accuracy of the simulation. Graph (d) of FIG. 14 illustrates an example Lyapunov loss of the simulation. FIG. 15 is a series of graphs (a)-(d) which illustrate four example simulation outputs for non-IID data, according to embodiments of the present disclosure. In this ^{example, accuracy/loss are shown with different strategies on non-IID data for y =} _{10. Graph (a) of FIG. 15 illustrates an example accuracy output of the simulation.} Graph (b) of FIG. 15 illustrates an example loss output of the simulation. Graph (c) of FIG. 15 illustrates an example Lyapunov accuracy of the simulation. Graph (d) of FIG. 15 illustrates an example Lyapunov loss of the simulation. The graphs of FIG. 14 and FIG. 15 plot the accuracy and loss with respect to cumulative time for y = 5 and y = 10. In each of FIG. 14 and FIG. 15, the first row (Graphs (a) and (b) of each of FIG. 14 and FIG. 15) is for the comparison among PCSRA and three baselines of separate optimization while the second row (Graphs (c) and (d) of each of FIG. 14 and FIG 15) is for the comparison among joint optimizations. It may be observed that PCSRA outperforms all three baselines of separate optimization and Dynamic-PCSRA outperforms all other joint optimization strategies, of a larger gain with a larger y. The curves in the non-IID case appear to be more “zigzag”-like in general, as compared to the IID case, illustrating that the values of p_' and G^' _< may vary substantially. Similar to the IID case, it may be observed that when y = 10, the three baselines of separate optimization collapse as the optimal solution is choosing each client 23 with probability one at all rounds. However, it may also be observed that the performance comparison among the joint optimization strategies can be seen more clearly in the non-IID case. The performance increases from the naive joint optimization, which does not consider the statistical heterogeneity, to only considering data ratios, to considering both data ratios and the importance of data, to eventually using the norms of the gradients dynamically. Examples of benefits to implementing embodiments of the present disclosure Implementing machine learning at the network edge (e.g., at a network node 16 and/or WD 22), and minimizing the allocation of limited resources, may be of great interest to telecommunications companies and network operators. In the present disclosure, a new convergence bound for non-convex loss functions is derived under the probabilistic client sampling framework, including a formulation of a general framework that jointly considers sampling and resource allocation with long-term and short-term constraints via stochastic network optimization, and which may be implemented by, e.g., a server 19 and/or client 23 in a communication system 10. Several novel per-slot probabilistic client sampling strategies are described herein which take into account resource allocation, in some cases with provable guarantees. A new framework and algorithms are described including examples of power allocation for communications and frequency allocations for computation. Embodiments of the present disclosure may thus provide one or more benefits such as: - Enabling and supporting configurations for machine learning at the network edge and in end user devices for a variety of learning tasks, e.g., under a joint probabilistic sampling and resource allocation framework described herein. - Enabling and supporting configurations for customizing different types of resource allocation based on the actual services being provided, which may include a variety of applications including but not limited to the examples described herein related to power allocation and CPU frequency allocation. For example, in addition to applications in conventional mobile broadband networks, other possible applications may include latency sensitive use cases, industrial IoT sensor networks, etc. - Enabling and supporting configurations for tailoring the long-term and short-term constraints of federated learning based on the actual machine learning tasks being performed and the system specifications for the system 10 performing the federated learning. By configuring properly defined constraints, the resources may be allocated optimally under such constraints, which may advantageously reduce costs due to resource utilization, e.g., as compared to unconstrained allocations or casual allocations. As will be appreciated by one of skill in the art, the concepts described herein may be embodied as a method, data processing system, computer program product and/or computer storage media storing an executable computer program. Accordingly, the concepts described herein may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or “module.” Any process, step, action and/or functionality described herein may be performed by, and/or associated to, a corresponding module, which may be implemented in software and/or firmware and/or hardware. Furthermore, the disclosure may take the form of a computer program product on a tangible computer usable storage medium having computer program code embodied in the medium that can be executed by a computer. Any suitable tangible computer readable medium may be utilized including hard disks, CD-ROMs, electronic storage devices, optical storage devices, or magnetic storage devices. Some embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, systems and computer program products. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer (to thereby create a special purpose computer), special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer program instructions may also be stored in a computer readable memory or storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. It is to be understood that the functions/acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows. Computer program code for carrying out operations of the concepts described herein may be written in an object oriented programming language such as Python, Java® or C++. However, the computer program code for carrying out operations of the disclosure may also be written in conventional procedural programming languages, such as the "C" programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, all embodiments can be combined in any way and/or combination, and the present specification, including the drawings, shall be construed to constitute a complete written description of all combinations and subcombinations of the embodiments described herein, and of the manner and process of making and using them, and shall support claims to any such combination or subcombination. Abbreviations that may be used in the preceding description include: FL Federate Learning SGD Stochastic Gradient Descent IID Independent and Identically Distributed Non-IID Not Independent and Identically Distributed BS Base Station CPU Central Processing Unit RS Random Scheduling RR Round Robin PCSRA Probabilistic Client Selection and Resource Allocation Algorithm PF Proportional Fair It will be appreciated by persons skilled in the art that the embodiments described herein are not limited to what has been particularly shown and described herein above. In addition, unless mention was made above to the contrary, it should be noted that all of the accompanying drawings are not to scale. A variety of modifications and variations are possible in light of the above teachings without departing from the scope of the following claims.

Claims

What is claimed is: 1. A server (19) configured to communicate with a plurality of clients (23) in a wireless communication system, the server (19) comprising processing circuitry (98) configured to: initialize an aggregate machine learning model for federated learning; and perform a federated learning on the aggregate machine learning model with the plurality of clients (23) for a plurality of rounds, a round of the federated learning including: determining an optimal subset of the plurality of clients (23); causing transmission, to at least the optimal subset of the plurality of clients (23), of a first indication of a current state of the aggregate machine learning model; receiving, from the optimal subset of the plurality of clients (23), a plurality of second indications corresponding to local machine learning models associated with the optimal subset of the plurality of clients (23); and updating the aggregate machine learning model by aggregating the local machine learning models.

2. The server (19) of Claim 1, wherein the processing circuitry (98) is further configured to: cause transmission of an initial configuration indication to the plurality of clients (23), the initial configuration indication including at least one of: a local learning rate; and a number of local steps.

3. The server (19) of any one of Claims 1 and 2, wherein the processing circuitry (98) is further configured to: initialize the aggregate machine learning model by initializing at least one virtual queue; and perform the federated learning by updating the at least one virtual queue based on the local machine learning models.

4. The server (19) of any one of Claims 1-3, wherein the processing circuitry (98) is further configured to: for each round of the federated learning, cause transmission, to all of the plurality of clients (23), of the first indication of the current state of the aggregate machine learning model; for each round of the federated learning, receive, responsive to the first indication, a plurality of third indications of a plurality of stochastic gradients associated with the plurality of clients (23); and for each round of the federated learning, the determining of the corresponding optimal subset of the plurality of clients (23) being based on the plurality of stochastic gradients.

5. The server (19) of any one of Claims 1-4, wherein the determining of the optimal subset of the plurality of clients (23) is based on a convex optimization of at least one of: radio channel resources associated with the plurality of clients (23); processing resources associated with the plurality of clients (23); and power resources associated with the plurality of clients (23).

6. The server (19) of any one of Claims 1-5, wherein the processing circuitry (98) is further configured to: receive radio measurement data associated with at least one client (23) of the plurality of clients (23); and predict a location of the at least one client (23) based on the radio measurement data using the aggregate machine learning model.

7. A method implemented in a server (19) configured to communicate with a plurality of clients (23) in a wireless communication system, the method comprising: initializing (S134) an aggregate machine learning model for federated learning; and performing (S136) a federated learning on the aggregate machine learning model with the plurality of clients (23) for a plurality of rounds, a round of the federated learning including: determining an optimal subset of the plurality of clients (23); causing transmission, to at least the optimal subset of the plurality of clients (23), of a first indication of a current state of the aggregate machine learning model; receiving, from the optimal subset of the plurality of clients (23), a plurality of second indications corresponding to local machine learning models associated with the optimal subset of the plurality of clients (23); and updating the aggregate machine learning model by aggregating the local machine learning models.

8. The method of Claim 7, wherein the method further comprises: causing transmission of an initial configuration indication to the plurality of clients (23), the initial configuration indication including at least one of: a local learning rate; and a number of local steps.

9. The method of any one of Claims 7 and 8, wherein the method further comprises: initializing the aggregate machine learning model by initializing at least one virtual queue; and performing the federated learning by updating the at least one virtual queue based on the local machine learning models.

10. The method of any one of Claims 7-9, wherein the method further comprises: for each round of the federated learning, causing transmission, to all of the plurality of clients (23), of the first indication of the current state of the aggregate machine learning model; for each round of the federated learning, receiving, responsive to the first indication, a plurality of third indications of a plurality of stochastic gradients associated with the plurality of clients (23); and for each round of the federated learning, the determining of the corresponding optimal subset of the plurality of clients (23) being based on the plurality of stochastic gradients.

11. The method of any one of Claims 7-10, wherein the determining of the optimal subset of the plurality of clients (23) is based on a convex optimization of at least one of: radio channel resources associated with the plurality of clients (23); processing resources associated with the plurality of clients (23); and power resources associated with the plurality of clients (23).

12. The method of any one of Claims 7-11, wherein the method further comprises: receiving radio measurement data associated with at least one client (23) of the plurality of clients (23); and predicting a location of the at least one client (23) based on the radio measurement data using the aggregate machine learning model.

13. A client (23) configured to communicate with a server (19) in a wireless communication system including a plurality of additional clients (23) in communication with the server (19), the client (23) comprising processing circuitry (84) configured to: at least one of receive, store, and determine a dataset for training a local machine learning model; perform a federated learning for an aggregate machine learning model with the plurality of additional clients (23) for a plurality of rounds, a round of the federated learning including: receiving, from the server (19), a first indication of a current state of the aggregate machine learning model; training, in parallel with an optimal subset of the plurality of additional clients (23), a local machine learning model based on the current state of the aggregate machine learning model and the dataset; and causing transmission, to the server (19), of a second indication corresponding to the local machine learning model.

14. The client (23) of Claim 13, wherein the processing circuitry (84) is further configured to: receive an initial configuration indication from the server (19), the initial configuration indication including at least one of: a local learning rate; and a number of local steps; and the training of the local machine learning model being based on the local learning rate and the number of local steps.

15. The client (23) of any one of Claims 13 and 14, wherein the processing circuitry (84) is further configured to: for each round of the federated learning, receive the first indication of the current state of the aggregate machine learning model; for each round of the federated learning, determine a stochastic gradient based on the local machine learning model; and for each round of the federated learning, cause transmission, responsive to the first indication, of a third indication of the stochastic gradient, the optimal subset of the plurality of clients (23) being determined based on the stochastic gradient.

16. The client (23) of any one of Claims 13-15, wherein the optimal subset of the plurality of clients (23) is determined based on a convex optimization of at least one of: radio channel resources associated with the plurality of clients (23); processing resources associated with the plurality of clients (23); and power resources associated with the plurality of clients (23).

17. The client (23) of any one of Claims 13-16, wherein the processing circuitry (84) is further configured to: cause transmission, to the server (19), of radio measurement data associated with at least one client (23) of the plurality of clients (23); and receive a location prediction based on the radio measurement data and the aggregate machine learning model.

18. A method implemented in a client (23) configured to communicate with a server (19) in a wireless communication system including a plurality of additional clients (23) in communication with the server (19), the method comprising: at least one of receiving, storing, and determining (S138) a dataset for training a local machine learning model; performing (S140) a federated learning for an aggregate machine learning model with the plurality of additional clients (23) for a plurality of rounds, a round of the federated learning including: receiving, from the server (19), a first indication of a current state of the aggregate machine learning model; training, in parallel with an optimal subset of the plurality of additional clients (23), a local machine learning model based on the current state of the aggregate machine learning model and the dataset; and causing transmission, to the server (19), of a second indication corresponding to the local machine learning model.

19. The method of Claim 18, wherein the method further comprises: receiving an initial configuration indication from the server (19), the initial configuration indication including at least one of: a local learning rate; and a number of local steps; and the training of the local machine learning model being based on the local learning rate and the number of local steps.

20. The method of any one of Claims 18 and 19, wherein the method further comprises: for each round of the federated learning, receiving the first indication of the current state of the aggregate machine learning model; for each round of the federated learning, determining a stochastic gradient based on the local machine learning model; and for each round of the federated learning, causing transmission, responsive to the first indication, of a third indication of the stochastic gradient, the optimal subset of the plurality of clients (23) being determined based on the stochastic gradient.

21. The method of any one of Claims 18-20, wherein the optimal subset of the plurality of clients (23) is determined based on a convex optimization of at least one of: radio channel resources associated with the plurality of clients (23); processing resources associated with the plurality of clients (23); and power resources associated with the plurality of clients (23).

22. The method of any one of Claims 18-21, wherein the method further comprises: causing transmission, to the server (19), of radio measurement data associated with at least one client (23) of the plurality of clients (23); and receiving a location prediction based on the radio measurement data and the aggregate machine learning model.