EP4792251A1 - Methods to enable federated learning deployment services - Google Patents

Abstract

Systems and methods for enabling federated learning deployment services are disclosed, comprising receiving a message comprising deployment configuration information for a trained ML model, wherein the deployment configuration information comprises policies for re-training the trained ML model. A request may be sent, to a first group of client devices, to deploy the trained ML model at the first group of client devices and a request may be sent to receive responses of deployment information associated with deploying the trained ML model. A response comprising the deployment information may be received from at least one of the first group of client devices. A determination to trigger re-training of the ML model may be made based on the one or more policies for re-training the ML model and the received deployment information. The method may comprise sending, to a second group of client devices, a request to re-train the ML model.

Description

METHODS TO ENABLE FEDERATED LEARNING DEPLOYMENT SERVICES

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Patent Application Number 63/589,883, filed on October 12, 2023.

BACKGROUND

[0002] Currently, existing federated learning (FL) service frameworks have a limited set of capabilities and only provide basic services to assist with training a machine learning (ML) model in a federated manner. The available level of functionality is suitable for proof-of-concept level implementations. However, FL frameworks currently lack critical functionality needed to deploy and manage FL-based systems at scale in commercial settings. For FL to become a mainstream technology, these FL service frameworks must first be enhanced with additional capabilities to ease complexity and enhance the robustness of FL such that the technology and its deployment may become viable.

SUMMARY

[0003] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is the summary intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to limitations that solve any or all disadvantages noted in any part of this disclosure.

[0004] This disclosure generally defines deployment centric capabilities for a federated learning (FL) service such that the FL deployment service may be deployed in systems requiring high levels of scalability and robustness. The disclosure defines functionality to enable a FL deployment service with the capability to receive FL deployment configurations from FL users, process and manage the deployment of ML models on behalf of FL users via interaction with FL clients, and update FL users with the status of the deployment process as the ML models are being used by the FL clients. [0005] The disclosure specifically describes a FL deployment service capable of, in part: 1) Receiving a message from a FL user comprising one or more types of FL deployment configuration information; 2) Based on the received deployment configuration information, performing one or more operations; 3) Sending a message of deployment configuration information to a FL client comprising one or more types of information; 4) Receiving a message of deployment report from a FL client comprising one or more types of information; 5) Based on the information received from the FL clients, determining whether to trigger model retraining and performing one or more operations; 6) Sending a message to the FL user comprising one or more types of information; and 7) Sending a message of updated deployment configuration information to a FL client comprising one or more types of information.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 shows an example system.

[0007] Figure 2 shows an example system.

[0008] Figure 3 shows an example system.

[0009] Figure 4 shows an example method.

[0010] Figure 5 shows an example method.

[0011] Figure 6 shows an example method.

[0012] Figure 7 shows an example method.

[0013] Figure 8 shows an example system.

[0014] Figure 9 shows an example system.

[0015] Figure 10 shows an example system.

[0016] Figure 11 A shows an example communications system.

[0017] Figure 1 IB shows an example apparatus configured for wireless communications.

[0018] Figure 11C shows an example system.

[0019] Figure 1 ID shows an example system.

[0020] Figure 1 IE shows an example system.

[0021] Figure 1 IF shows an example system.

[0022] Figure 11G shows an example system.

DETAILED DESCRIPTION [0023] Methods and apparatuses are described herein for enabling federated learning deployment services.

[0024] The following abbreviations may be used herein:

[0025] The following definitions may be used herein:

[0026] federated learning user (FL user) is a user apparatus (e.g., computer, smart phone, tablet) hosting compute, memory, and network connectivity resources capable of hosting a user application software (e.g., web browser, app) which a user may use to perform FL centric operations with a FL deployment service. For example, an FL user may configure an FL deployment service with FL deployment requirements such as FL cluster and ML model selection and configuration policies.

[0027] federated learning client (FL client): A network apparatus (e.g., device, gateway, or server) hosting compute, memory, and network connectivity resources capable of hosting a FL client software (e.g., FL client application) and local data which the FL client may use to perform operations such as training a ML model in a federated manner or inferencing using the trained ML model.

[0028] federated learning deployment service (FL deployment service): A network apparatus (e.g., device, gateway, or server) hosting compute, memory, and network connectivity resources capable of hosting FL deployment service software which may perform FL management operations such as training a ML model in a federated manner across FL clients.

[0029] federated learning training round (FL round): One iteration of training performed on a ML model in a federated manner by FL clients wherein the training is managed by a FL deployment service. [0030] federated learning task (FL task): A set of instructions and requirements sent by a FL deployment service to multiple FL clients for training a ML model in a federated manner. A FL task may be generated by the FL deployment service for each round of federated training that is performed on an ML model. A FL task may be specified by a FL user or a FL deployment service.

[0031] Federated Learning

[0032] The rapid advancement and expansion of ever increasingly compute capable smart mobile phones, Internet of Things (loT) and similar devices has led to an exponential growth of data being generated at the network edge. This poses new challenges for conventional cloudbased centralized approaches for data analysis, as these approaches are not scalable or efficient enough to handle the large volumes of data generated by the vast volume of edge client devices. Additionally, these approaches raise privacy concerns, as the data collected from edge devices could be shared with third parties.

[0033] To address these challenges, federated learning (FL) has recently come to the forefront to address the need for a distributed solution that preserves privacy for data-driven learning and inference-based applications to enhance efficiency and address privacy concerns. FL is starting to gain interest for use cases such as healthcare, finance, and transportation. In FL, the training of machine learning (ML) models for data-driven applications is a collaborative effort among distributed clients, without the need to centralize client data. This helps to protect user privacy and improve the efficiency and scalability of data analysis. The decentralized and cooperative nature of federated learning aligns perfectly with the network edge, where each device may act as an individual client. Furthermore, since raw data gathered by each client / device isn't shared with others, FL offers an effective way to safeguard user privacy, where client devices may directly gather data comprising sensitive personal information.

[0034] Federated Learning (FL) is an approach to machine learning (ML) that enables the creation of models on data that is distributed across multiple clients or locations, without the need to aggregate and store data in one central location. An FL service manages multiple FL clients during ML training and/or inferencing and data is kept private and stored locally at the individual FL clients. ML model(s) and the associated parameters are exchanged between the FL service and the FL clients during training. The FL service aggregates the model parameters (e.g., weights) at each round of federated training and the FL training iterates for many rounds. This method has become increasingly popular in recent years as more and more organizations seek to harness the power of large datasets without compromising the privacy of individual users and/or data sources, with the added benefits of reduced data movement, and improved scalability.

[0035] Currently, existing FL service frameworks have a limited set of capabilities and only provide basic services to assist with training a ML model in a federated manner. This level of functionality is suitable for proof-of-concept level implementations. However, FL frameworks currently lack critical functionality needed to deploy and manage FL-based systems at scale in commercial settings. For FL to become a mainstream technology, these FL service frameworks must first be enhanced with additional capabilities to ease complexity and enhance the robustness of FL such that the technology and its deployment may become viable.

[0036] There are several challenges and shortcomings to existing FL service frameworks that may need to be addressed before FL may be widely adopted and deployed at scale. In this disclosure, the following challenges are addressed:

[0037] 1) Diversity of FL deployment bandwidth (BW) availability at the edge: The network bandwidth available at the edge is often limited but may also have variability across FL clients. The limited and/or variable network bandwidth may cause difficulty in deploying a single, one-size-fits-all “master” FL model to all clients. Because different ML algorithms, with different sets of parameters, hyperparameters, and/or software dependencies, may consume different levels of network resources in the process of communicating the model parameters.

[0038] 2) Diversity of FL client data: The diversity of FL clients' data may present a number of challenges for FL. For example, clients having different distributions of data may cause difficulty in training a global model that generalizes well to all clients. The clients may also have different biases or different levels of noise or uncertainty in their data which may further complicate training a reliable global model.

[0039] 3) Diversity of available FL client resources (a.k.a. heterogeneity): Like in BW availability, FL clients may also show a wide range of available computational resources. The FL clients may have different hardware and/or software (dependencies), which may cause difficulty in designing FL algorithms that are equally efficient and scalable for all FL clients.

[0040] 4) Intermittent FL client connectivity and availability: FL clients may often be disconnected from the network or new FL clients may join/leave intermittently. This variability in FL client availability needs to be managed by a central entity. [0041] 5) Temporal data dynamics after deployments (a.k.a. drift): The data distribution on FL clients may change over time, which leads to data drift. In general, drift in ML refers to changes in the distribution of data over time. This may happen for a variety of reasons, such as changes in the underlying population, changes in the way data is collected, or changes in the way data is processed. Drift may negatively impact the performance of a trained machine learning model. This is because the model is trained on a specific distribution of data, and if the data distribution changes, the model may no longer be able to accurately predict the outcome. Thus, drift may cause difficulty in maintaining the accuracy of the FL models. Of course, an additional dimension in FL is the variety and volume of clients, i.e. different (groups of) clients’ data may vary as well.

[0042] The present defines deployment centric capabilities for a federated learning (FL) service such that the FL deployment service may be deployed in systems requiring high levels of scalability and robustness. The disclosure defines functionality to enable a FL deployment service with the capability to receive FL deployment configurations from FL users, process and manage the deployment of ML models on behalf of FL users via interaction with FL clients, and update FL users with the status of the deployment process as the ML models are being used by the FL clients.

[0043] Specifically, the disclosure defines:

[0044] A FL deployment service capable of:

[0045] 1) Receiving a message from a FL user comprising one or more of the following types of FL deployment configuration information:

[0046] a) The need/goal and use case for the FL deployment;

[0047] b) The features and typical algorithms associated with the FL deployment, outcome variable/entity to be predicted;

[0048] c) A list of FL clients for the deployment and information of each of the FL clients, such as:

[0049] i. Identifier of the FL client

[0050] ii. Device/entity type of the FL client

[0051] iii. Supported role (training, inferencing, both) of the client.

[0052] iv. Functionalities supported at the FL client, such as data processing/preparation, ability to collect ground truth, model performance measurement/evaluation, etc. [0053] v. Information of the data that is or may be available at the FL client, such as the metadata and/or statistics of the data.

[0054] vi. Information of the resources and capabilities available at the FL client for FL deployment, such as the status and/or schedule of computational resources, (battery) power, memory/storage, network/communication bandwidth, mobility, etc.

[0055] d) Requirements and/or filter criteria of the models to be deployed, such as:

[0056] i. Identifier of the model

[0057] ii. Required/expected performance of the model

[0058] iii. Required/preferred ML algorithms and/or hyperparameters associated with the model

[0059] e) Model usage policy indicating when a FL client is allowed to use the deployed ML model

[0060] f) Evaluation methods or performance metrics of the deployed ML model

[0061] g) Instructions to the FL clients on what information should be tracked and/or reported to the FL deployment service

[0062] h) Instructions to the FL deployment service on what information should be tracked and/or reported to the FL user

[0063] i) Clustering policy of the FL clients, such as when/how to perform clustering, the algorithm to build clusters, criteria for clustering

[0064] j) Model retraining policy indicating when the FL deployment service should trigger model retraining and what actions are to be taken by the FL deployment service when model retraining is triggered.

[0065] 2) Based on the received deployment configuration information, performing one or more of the following operations:

[0066] a) Performing clustering of the FL clients based on their supported roles (willingness), data property, device/entity property (computational capability, network resource, power constraints, mobility), or a combination thereof.

[0067] b) Discovering a pre-trained ML model based on the model requirements and filter criteria.

[0068] c) Determining the deployment configuration for each FL client or each cluster of clients. [0069] 3) Sending a message of deployment configuration information to a FL client comprising one or more of the following types of information:

[0070] a) (Identifier of) the discovered pre-trained ML model to be deployed at the client

[0071] b) Identifier of the cluster that the client belongs to (and the role assigned to the client)

[0072] c) Model usage policy

[0073] d) Evaluation methods or performance metrics of the deployed model

[0074] e) Instructions on what information should be tracked and/or reported to the FL deployment service and the corresponding reporting schedule

[0075] f) An indicator of whether model retraining may be triggered by the client, if so, the corresponding trigger conditions and the actions to be taken by the client when retraining is triggered.

[0076] 4) Receiving a message of deployment report from a FL client comprising one or more of the following types of information:

[0077] a) Statistics/metadata/characteristics of the data (used for training or inferencing) at the FL client

[0078] b) Performance evaluation of the ML model deployed to the FL client

[0079] c) Resource or time consumption information of using the ML model at the FL client

[0080] d) Deployment related events detected at the FL client (e.g. data/model drift, resource shortage)

[0081] 5) Based on the information received from the FL clients, determining whether to trigger model retraining and performing one or more of the following operations:

[0082] a) Determining whether model retraining should be triggered for a single FL client

[0083] b) Aggregating the information received from FL clients in a FL cluster and determining whether model retraining should be triggered for the FL cluster

[0084] c) Aggregating the information received from more than one FL client and determining whether model retraining should be triggered

[0085] d) Determining the existing ML model should be retrained/updated, or be replaced by another pre-trained model, or be replaced by a new model yet to be trained [0086] e) Updating the clustering of the FL clients

[0087] 6) Sending a message to the FL user comprising one or more of the following types of information:

[0088] a) Aggregated information from the FL clients

[0089] b) A notification that model retraining is triggered and the corresponding operations performed by the FL deployment service or FL clients

[0090] c) A request to ask the FL user to determine if model retraining should be triggered

[0091] d) Updated clustering information of the FL clients

[0092] e) A request to ask the FL user to update the clustering of the FL clients

[0093] 7) Sending a message of updated deployment configuration information to a FL client comprising one or more of the following types of information:

[0094] a) (Identifier of) the updated/new ML model to be deployed at the client

[0095] b) Updated identifier of the cluster that the client belongs to (and the updated role assigned to the client)

[0096] c) A request to train/update the ML model deployed at the client

[0097] d) (Updated) model usage policy, evaluation methods, reporting instructions, etc.

[0098] This disclosure defines advanced features of an FL deployment service for enabling more scalable and robust FL deployments comprising FL clients having heterogonous data and computational/networking capabilities.

[0099] FL Client Clustering

[0100] One important aspect of the disclosure is that clusters of FL clients may be dynamically formed due to client heterogeneity in multiple dimensions for and during FL deployment. These “clusters” may be formed, via functionality supported by a FL deployment service, to address for example heterogeneity in participation agreement, data characteristics, computational and network capabilities, power constraints, mobility status, and temporal schedules of different FL clients. In turn, separate algorithms and/or features (as in vertical FL) may be managed by an FL deployment service on a per cluster basis. For example, horizontal FL wherein selected portions of data from different clients (in each client cluster if applicable) may complement each other in the training process. [0101] To tackle potential dynamic heterogeneities of FL clients, multiple clustering techniques may be supported by an FL deployment service to divide the FL clients into different clusters in one or more dimensions. For example, different ML algorithm or model may have requirements on unique characteristics of data, computational, and/or network resources. Clusters with low computational resources or limited network bandwidth might be more suited to simpler, less computationally intensive models like logistic regression or decision trees. On the other hand, clusters with high computational resources and good network connectivity might be able to handle more complex models like deep neural networks. Moreover, not all clients may have the resources to train models. Some clients may only be able to host models for inference purposes. In this case, a hybrid approach may be supported by an FL deployment service where some clients (typically those with more computational resources) are assigned for training the models, while others are assigned to use these trained models for inferencing purposes. Also, to distribute training and inference clients as equally as possible across clusters, an FL deployment service may use a load balancing strategy. This strategy may ensure that each cluster has a fair distribution of both types of clients. Load balancing performed by an FL deployment service may take into consideration across various factors such as the computational load of each client, network usage, and data similarity.

[0102] An FL deployment service may support one or more of the following FL cluster management features:

[0103] 1) Filtering of clients based on participation agreement: There may be a willingness identifier that may be determined by FL clients (either by the end-user of FL clients or automatically by FL clients). The willingness identifier may take effect at the FL deployment initiation stage, but the identifier may also dynamically change during the iterative process, as described in further steps of the procedural flow.

[0104] 2) Filtering of clients based on power and mobility: Power constraints and mobility status may help determine if or how long clients may be operational in a FL process.

[0105] 3) Clustering of clients based on data characteristics: Clusters of FL clients may be created based on data statistics, metadata and other information related to FL user’s and/or FL clients’ data of interest for the ML use case at hand. The clusters of FL clients may be called “Data Clusters” (DC). The management of these clusters may determine ML algorithms, data preparation and other relevant ML pipeline steps and/or components for the clients in each FL cluster.

[0106] 4) Clustering of clients based on computational capabilities, network capabilities, power constraints, mobility, etc.: Computational and network capabilities may determine if clients may be assigned to training or inference clusters.

[0107] 5) Clustering of clients based on time or schedule of the use case and/or clients: for example, clusters may be formed/reformed in different parts of the year depending on the seasonality properties of the use-case at hand (sales, finance, etc.).

[0108] Figure 1 shows an example how the filtering and partitioning processes may be manifested by an FL deployment service in general. FL client clusters may be formed according to requirements from a particular deployment and hence by a particular ML model. Using these requirements, an FL deployment service may evaluate FL clients on various clustering criteria as shown in Figure 1. In the example, FL client cluster 1 is formed based on data characteristics, computational and network capabilities, and available schedules for the FL deployment. FL client cluster 2, on the other hand, may require a different set of clustering criteria, mainly the data characteristics, networking capability, power constraints, and location and mobility. The clustering criteria may be performed in any order and the priority of the criteria may be dependent on the requirements of the FL deployment.

[0109] Central to any clustering mechanism is the need to align characteristics of available data on FL clients to the requirements of the ML model. This matching is imperative to ensure FL deployments are using data applicable to the ML model. Other clustering criteria may also be accounted for to ensure seamless FL deployments, such as computing and networking capabilities, power constraints, location and mobility, and time and scheduling availability of FL clients. Note that additional clustering criteria may also be envisioned to support the variety of FL deployments that may be encountered.

[0110] One criterion to consider may be the “willingness” of FL clients to participate in the FL deployment. Some FL clients may “turn off’ their willingness by default, some may change their decision on an ad-hoc basis. Therefore, client participation may be a dynamic property which the FL deployment service is provided with (either by the FL clients and/or by an FL user). Clients may be filtered based on their power constraints and mobility status. The FL deployment service may factor the power source of the clients and determine the FL clients that

-l i are able to perform the required FL task. For example, an FL client with a battery source may only be available for a finite duration of time and hence, the FL deployment service may only schedule one or two rounds of FL training or cluster the FL client for inferencing only. Similar decisions may be made based on the mobility status of FL clients, comprising the location of the FL client. Analyzing locations, accelerations, speeds and directions of motions of clients, a FL server may make decisions as to if or to what extent the clients may be able to participate in the FL process.

[0111] Figure 2 shows a scenario how the series of filtering and clustering steps may be implemented to concretely clarify the concept, where (a) shows a sample scenario wherein FL client clustering occurs per data statistics and metadata and computational and network capabilities in a cascaded manner and (b) shows a detailed view of distribution of data based (sub)clusters to training and inference clusters in this scenario during CNC based clustering. In Figure 2 (a), once initial filtering based on participation agreements, power resources and mobility status of all clients in the network are completed and the participating and eligible FL clients are determined each iteration, a data statistics based clustering process may be started that a FL deployment service may support. This process may be based on statistics and metadata of the FL clients’ data only. This segmentation of clients may determine the type of algorithm selection as well as data preparation steps suitable for a given cluster of clients. Hence, there may be multiple “master” models for the FL deployment suitable for different “Data clusters” (DC) (e g., DC 1, DC 2, etc.).

[0112] As a final clustering step, a “Computational and Network Capability” (CNC) clustering may be applied to the clients that were put in different data clusters previously. In this step, computational capabilities such as CPU, storage, memory and/or network bandwidth, speed, etc. may determine the extent to which FL clients may be able to participate in the FL deployment. In the scenario of Figure 2, this selection is shown to be made as a binary decision (Training and Inference), but other approaches may also be followed wherein different clusters may be created by the FL deployment service according to computational capabilities of the FL clients. For example, Level 1 may designate an “inference-only” cluster with FL clients having limited computational capabilities, as training typically takes more computational resources than inference. As the computational capabilities of FL clients increase, so may the degree to which training participation may occur. The FL clients in the highest tier of computational capability may be used for ML training only (or for both training and inference).

[0113] Sticking with the binary selection of the scenario depicted in Figure 2, it should be noted that the computationally capable clients would be able to carry out training as well as inference. Because training is in general a computationally more demanding process, clients deemed to be able to carry out training would also be able to participate in the inference process. In addition, it is important to recognize that there must be a one-to-one correspondence between DCs in training and inference clusters (as subclusters) and should be somewhat proportionally represented in each. In Figure 2 (a), a scenario is shown wherein there are 100 clients in DC 1, 80 clients in DC 2, and 40 in DC N (n¹¹¹ DC) in the training cluster. In terms of the said proportionality requirement, an ideal inference cluster configuration may involve DC 1 with 120, DC 2 with 96, and DC N with 48 FL clients (Each DC in the inference cluster is 6/5^th of their counterparts in the training cluster in terms of the numbers of FL clients in them). The distribution process of DC clients in the CNC clustering process is also shown in Figure 2 (b) in a bit more detail. In essence, the “strong”, or computationally and network-wise more capable clients in each DC would be grouped in a subcluster under the same DC label, in the training cluster. The “weak” FL clients would be similarly grouped within the inference cluster.

[0114] Accordingly, it’s worth to emphasize that; computational and network capability related properties of the FL clients are in general not related to the characteristics of the data of interest for the ML use case they’re generating and/or storing. Therefore different “Data Clusters” may be distributed by the FL deployment service to the training and inference clusters proportionally so each data cluster may be equally represented in the training and inference clusters. A different set of ML-processes and entities (data preparation, cleaning, algorithm, etc.) may be utilized for the different clients in different data clusters. For example, a ML model that was trained on the FL clients in the training cluster (realized via computational and network properties) and that also belongs to the subcluster “Data Cluster 1” (realized via data properties) must be deployed to clients in the inference cluster that are also part of the same Data Cluster 1. This is shown in Figure 3. The reasons may be multitude. Firstly, the inference process may comprise the same data transformation steps as the ML training process which are dictated by the data clustering process proposed. In addition, different ML algorithms may be deemed suitable for the data of the clients in different data clusters, and there must be a match of data subclusters between the clients in training and inference clusters. If, on the other hand, a certain data subcluster in the training cluster cannot be formed in the inference cluster (or vice versa) due to the lack of clients with the same data characteristics of that FL cluster in either training/inference cluster, those clients may be left out of the FL process. Alternatively, the clients may be merged with another, “similar” data cluster that may be otherwise available and similarly represented in each training and inference cluster.

[0115] FL Deployment Service Procedure

[0116] The general method of FL deployment service is shown in the call flow diagram of Figure 4. The method describes an FL deployment service managing FL client clusters during FL deployment for ML model training and inference. Note that the internal cluster structure of FL clients is not shown in Figure 4 (which may be found in Figure 1, Figure 2, or Figure 3). Note that the described steps and method may be performed in an order different than what is shown in the figure.

[0117] 1) Configuration of the FL deployment service: A FL user may define the requirements for an FL deployment. This could be anything from disease detection to personalized recommendations based on the location of mobile phones. The user may also specify the features and typical algorithms to achieve the goal of the FL deployment. Subsequently the FL user may start a FL deployment by interacting with an FL deployment service (e.g., via a GUI or API), and may specify a list of FL clients. This list may be defined as FL clients in a particular deployment/cluster. This step may also involve communications between the FL deployment service and the FL clients in the entire system with regards to FL client’s willingness to participate in FL deployments which may filter out the unwilling FL clients in the system. The FL deployment service may also recommend which FL clients are most suitable for the FL deployment among the ones that are willing to participate.

[0118] The FL user(s) may provide FL client information to the FL deployment service so that the FL clients may be grouped into appropriate FL client clusters for model deployment. Table 1 shows examples of FL client information that may be provided to the FL deployment service by the FL user.

[0119] Table 1. FL Client Information

[0120] The FL user(s) may provide configuration information of the deployment to the FL deployment service, such as the requirements/preferences of the model to be deployed (filter criteria for model discovery), trigger condition of model re-training, etc. Alternatively, a model discovery procedure may be initiated by one or more FL cl ients/cl usters (such as described hereinafter in Figure 5). Table 2 shows examples of FL deployment configuration information that may be provided to the FL deployment service by the FL user.

[0121] Table 2. FL Deployment Configuration

[0122] The FL deployment service may group FL clients into training and inference clusters using information from Table 1 and Table 2. The clusters may be generic or specific depending on available deployment configurations. For example, the FL deployment service may group FL clients into global training and inferencing clusters initially based on received FL client information and form more specific training and inferencing sub-clusters based on model and data requirements, FL client schedule and availability, and FL computation capabilities specified in FL deployment configurations.

[0123] 2) The FL deployment service possibly in conjunction with assistance from ML services, and with or without the FL users’ involvement, may determine/locate an initial, pretrained, FL/ML model(s) for one or more participating FL clients. The model(s) may be determined based on the information provided by the FL user(s), such as the FL client information (Table 1) and the FL deployment configuration (Table 2).

[0124] 3) The initial FL/ML model may be deployed to one or more FL clients/clusters. This may start the FL operation, as in each practical use case, the clients may expect to start using the ML model for inferencing tasks immediately from the start of the procedure.

[0125] If step 1 had not performed FL client clustering operations, the FL deployment service (and/or the FL user) may determine the initial clustering of the FL clients based on the FL client information (e.g., supported roles of the clients). The model may be deployed to a FL client in the inference cluster to perform the inferencing process to generate inferencing results. The model may also be deployed to a FL client performing both training and inferencing processes, where the client may consume the model to generate inferencing results while contributing to the training of the model.

[0126] The FL deployment service may determine the per-client deployment configuration and send the information to the FL clients when deploying the model. The per- client deployment configuration may be determined/derived based on the information provided by the FL user(s), such as the FL client information (Table 1) and/or the FL deployment configuration (Table 2). For example, the per-client deployment configuration may send the instructions to a FL client on what information should be tracked and reported to the FL deployment service, whether the FL client is allowed to trigger model retraining, etc. The deployment configuration may also be specified per-cluster and information about the configuration may be sent to FL clients in the cluster(s) similar to the per-client scenario.

[0127] FL Deployment Loop: Steps 4 to 10 may be performed iteratively and/or periodically. The periodicities involved in this process may depend on factors such as the FL deployment configuration, frequency of data availability, dynamics in the training/inferencing data, model performance, inferencing results, status of the FL clients, etc.

[0128] 4) This step may comprise FL training as well as inferencing. These operations may occur simultaneously and not necessarily sequentially. Training may occur in FL clients capable of supporting the more computationally demanding training procedure, and which have been assigned to FL training cluster / sub-clusters by the FL deployment service. Training may be carried out by each FL client according to the conditions and requirements associated with the clusters that have been created as part of an FL deployment configuration. The FL clients in a training cluster may send updated model parameters to the FL deployment service based on completion of their local training (as in a regular FL operation but per their ML sub-cluster). Depending on different clusters that may have been determined, there may be separately grouped sets of updates for each cluster. An authorization request may be sent from the FL deployment service to FL user(s) with regards to the aggregated FL model(s), and a response may be received by the FL deployment service from the user(s) (not shown). The model parameter aggregation may be carried out per FL clusters. The FL aggregation algorithm may be configured in step 1, which may be dynamically modified due to different client clusters and/or the modifications thereof during the process. If new FL and cluster configurations were carried out in steps 7-10, step 4 may also entail the “deployment” of an initial models (per each FL cluster/subcluster). In this case, updated ML models may be deployed to both training and inference clusters per FL clusters/sub-clusters. The local training may continue in the training cluster clients using the aggregated model, which at the same time may be deployed to FL clients in the inference cluster to produce inference results as well. FL Inference may be carried out at FL clients in the inference cluster (as well as training cluster, as inference is less compute-heavy, hence may also be carried out in training cluster clients), according to their FL cluster/sub-cluster label. This may comprise the evaluation of the ML models for each FL cluster. [0129] 5) Model evaluation and performance reports may be communicated from the inference cluster clients to the FL deployment service. The reports may send the required information as indicated in “client reporting instructions” defined in Table 2. The reports may be sent periodically, based on a schedule, on-demand by request, or in an event-based manner. In the event-based case, the triggering of sending the reports at a FL client in the inference cluster may depend on the model accuracy. If the model accuracy drops below a preset threshold the model evaluation report may be sent to the FL deployment service. The FL deployment service may in turn send the model evaluation report or subset thereof to the FL user(s). The FL deployment service may present a continuously / periodically updated dashboard to the user(s), and the user may also demand a more detailed report from each client showing previously acceptable values of model accuracy and how the model has evolved over time. This dashboard may also comprise updates on the FL client information (data statistics, metadata, computational resources, network statistics). The triggering of the report may also depend on the FL information package of the next step via some predefined or adaptive thresholds. In addition, the “timeliness” of the model evaluation results may depend upon the speed at which the ground truth values of the FL deployment use case at hand may be produced at (and sent by) the inference clients.

[0130] The computational efficiency of a model may also change from time to time due to data characteristics, independent of FL client characteristics. Hence, compute time, as well as numbers of iterations and time for each iteration taken for the model to reach a convergence may also be part of the model evaluation report. Due to the nature of mathematical optimization inherent in any machine learning process, and various relevant parameters (such as optimization algorithm, learning rate, numbers, and types of layers in neural network algorithms, or other hyperparameters), the convergence may take longer in some cases. There may even be instances when convergence may never be reached. In such cases, some threshold-based policies may be used by the FL deployment service. For example, policies based on time elapsed per run or number of iterations may be utilized by the FL deployment service for early stopping or terminating some training processes and making modifications to the model.

[0131] 6) As part of the iterative process, the FL deployment service may request, from FL clients, relevant data statistics, meta data information, information about clients’ computational and network resources, and the clients may respond with the requested information. Based on the received information, the FL deployment service may update the FL client information (as found in Table 1). The updating may be done with the same frequency as the model inference results or less/more frequently. As part of this communication, FL clients may turn off/on participation as well. Consequently, new clients may appear on and/or existing clients may drop off from the FL deployment. The details of the data communication in step 6 are discussed in step 7. The computational and network capability information associated with FL clients may also be included.

[0132] The operations in steps 4 to 6 may also be realized as described hereinafter in the FL Client Reporting section.

[0133] 7) Model retraining and client cluster management:

[0134] Based on information received from Steps 5 and 6 and the model retraining policy provided by the FL user, the FL deployment service may determine if model retraining should be triggered and what actions are to be taken. Alternatively, the FL deployment service may aggregate information reported from FL clients and send the information to the FL user(s) to determine whether model retraining should be triggered.

[0135] For example, if a global trigger is defined as “model retraining is triggered when data/model drift is detected by at least 80% of the inferencing clients “, the FL deployment service may record the occurrences of data/model drift which may be reported by FL clients or detected by the FL deployment service based on the reports from FL clients. When the number of clients with data/model drift exceeds 80% of the total number of clients, the model retraining may be triggered.

[0136] As another example, a threshold may be specified such that when a certain number of FL clients report issues with an ML model’s performance, model retraining may be triggered. A FL deployment configuration may specify that if 10% of FL clients report that model performance falls under 95%, model retraining may be triggered. The threshold may be specified based on a particular client cluster or ML model.

[0137] Based on the information received in the previous step and/or the detection of model retraining triggers, the FL user(s) and/or the FL deployment service may make determinations with regards to re-clustering of FL clients. For example, data/model drift detected at FL client(s) in the inference cluster may indicate that the data at the client(s) may have patterns that are new to the existing model. In this case, the client(s) may be moved/added to the training cluster so that the client may contribute its data to improve the model. In another example, a client may participate in both training and inferencing. If the model performs badly at the client while performing well at other inferencing clients, the data at the client may be an outlier and the client may be removed from the training cluster.

[0138] If the compute and/or network resources of some clients are not sufficient, it may not be possible to move such clients to the training cluster. Instead, clients from the training cluster, that also belong to the same subcluster may be analyzed to see if the clients show similar changes in their data patterns. If so, a “cluster retraining” (new algorithms, new features, new data prep steps etc.) may be done for that cluster. If not, other FL clusters may be analyzed to see if this new data patterns in these inference clients fit to one of the other FL clusters, so these inference clients’ FL cluster label may be changed to those. Alternatively, new clusters from scratch may be formed as well.

[0139] During this step, the FL deployment service may re-configure clusters of clients based on their data, computational and network characteristics. This re-configuration of clusters may be based on information provided by the FL user to the FL deployment service. In addition, cluster management of FL clients may be a dynamic process, which may be modified according to changes in participation, data, and/or network patterns. The FL deployment service may determine that models for some clusters or individual FL clients may not be performing well. This determination may be made based on changes in data, or computational/network resource patterns associated with those clusters and/or clients. As a result, some clients may be assigned to different clusters, or removed from the FL System entirely due to deficiencies in the properties, or new/replacement clients may be added to the FL system. FL clients may also change their participation status during the lifetime of an FL deployment (by themselves or by the corresponding FL user). Alternatively new FL clusters may be added by the FL deployment service and/or FL user. For example, should some clients change their behavior due to “systemic” changes in their data, computational or network resources.

[0140] In some cases, if the updated version of a ML model (per cluster) is not producing satisfactory ML outcomes for some clients, the model retraining may be reverted (i.e., rolled back) by the FL deployment service so that such clients may continue using a previous version of the model for their FL cluster, while others receive the updated version. The model may be the same algorithm, but with different model parameters and/or hyperparameters. These individual differences may evolve to collective differences and may eventually result in new clusters being formed by the FL deployment service if enough clients fall into the same category.

[0141] It is also worth mentioning that the methodology through which the clustering may be realized, may differ in the following manner:

[0142] a) Full user involvement (manual): In this scenario, the FL user(s) may be fully involved in determining the training/inference as well as the FL clusters of FL clients. The FL deployment service may share the FL client information received from the FL deployment with the FL user(s). The user(s) may make an ad-hoc decision or apply a prebuilt recipe to create the FL clusters.

[0143] b) A preset policy of clusters based on client characteristics (rule-based, automatic): Alternatively, the FL users may configure a policy, either within the FL deployment service or within another service (e.g., a Policy Engine (PE) service not shown in the procedural diagram) which the FL deployment service interfaces with. The policy may be used to manage automatic assignment of FL clients to FL clusters.

[0144] c) ML based process (intelligent, automatic): The clustering of FL clients may be done through a FL clustering algorithm. The algorithm may be communicated as part of the initialization of the FL deployment service. Alternatively, the algorithm may also be modified by the FL user on an ad-hoc basis. The FL deployment service and/or FL user may also leverage one or more ML Services to make dynamic recommendations regarding the clustering decisions based on changes in FL client data and/or other characteristics.

[0145] Based on the results from the model evaluation reports and updated client information, model and/or data drifts may be detected (on a FL cluster basis) as explained previously. This determination may be made by the FL deployment service and/or the FL user(s). Consequently, clients may be assigned to different clusters and/or new clusters may emerge as described. In the latter case, new data preparation and/or ML algorithm requirements may be determined by the FL deployment service and/or user(s).

[0146] 8) If no model/data drift or other cluster-changing data pattern changes are detected by the FL deployment service, and no new clusters and/or new ML models are created/utilized as part of step 7, a simple model validation response may follow from the FL deployment service to FL clients, after which the iterative process may return to step 4. [0147] If, on the other hand, model/data drift is detected, and new clusters are formed through one of the aforementioned methods, the new cluster assignments may be sent to the FL clients, and new cluster operations may be carried out. As part of the FL cluster (and/or ML pipeline steps to be performed by the FL clusters) management process, cluster assignments may be sent to FL clients. Accordingly, the FL training and inference clusters that may also have FL sub-clusters may be constructed accordingly. FL cluster formation and management may be dynamically controlled by the FL deployment service resulting in changing the assignment of FL clients to FL clusters during the lifetime of an FL deployment.

[0148] If model retraining is triggered at step 7, the FL deployment service may determine if a new model is required (e.g. training a new model from scratch, or switching to another model based on the similarity measures) or the existing model should be updated (e.g. resuming the FL training process associated with the model, training the model with updated training data, adding fine-tuning process, applying transfer learning to the model, etc.). According to the result, the FL deployment service may perform actions such as those defined in a pre-configured global action. For example, the FL deployment service may request a new or updated model from an ML service or instruct FL clients in a training cluster to train/update the model. The updated model may be deployed to the FL clients. Depending on the configuration, the updated model may be deployed to FL clients in an inference cluster. Alternatively, the updated model may be deployed to only FL clients experiencing data/model drift, while the rest of the clients may continue using the old version of the model. The FL deployment service may track the deployment information of the model (e.g. which FL client is using which version of the model) and keep the information updated.

[0149] 9) After the different FL Training/Inference clusters have been updated, and their corresponding client assignments and any ML pipeline steps to be performed have been determined, the FL deployment service may also communicate with ML service(s) for relevant ML algorithms and other ML pipeline steps/components required for the specified FL deployment use case and which are suitable for each FL cluster. The ML service(s) may also communicate with storage services (not shown in Figure 4) to locate pretrained models that may be determined based on similarity measures between the pretrained model’s corresponding FL cluster’s client characteristics and those of an existing cluster. If a pretrained model is found, the ML service may send the relevant information to the FL deployment service. If a relevant pretrained ML model does not exist, the FL deployment service may send a request to the ML service to create a “vanilla” ML algorithm with randomly initialized parameters and hyperparameters. In either case, the ML service may also send data preparation as well as dependency requirements to the FL deployment service. The ML Services may also have feature, data preparation, or algorithm related recommendations based on the ML use case / client characteristics.

[0150] 10) If steps 7-9 were carried out involving new cluster formations, new initial (“vanilla”) FL models or models that are to be updated/retrained may be downloaded to the updated training cluster clients (per their updated FL cluster/subcluster). The FL deployment service may send an authorization request to and receive a response from the FL user(s) for the ML model (not shown) for the initial download of the models to the FL clients in the training cluster. The FL deployment service may send information relevant to the authorized ML model(s) to the FL clients in the training cluster, along with the steps to fulfill the dependencies, data preparation requirements, and training procedures. The information sent may comprise model specification parameters such as location at the ML Services, algorithm, model ID, version, training instructions with model parameters, HPO, as well as computational environment specifications (language, OS etc.). This step may also comprise model accuracy parameters (and the procedures to calculate them) and the acceptable ranges thereof for a local training to be deemed converged later in the procedure. This step may be realized through the FL user’s involvement through a GUI, a CLI editor connected to the FL deployment service, or a deployment script (e.g., a YAML or JSON file) as well. Based on the results, the iterative process may return to Step 4 and updated ML models may be deployed to both training and inference clusters per FL subclusters. The local training may continue in the training cluster clients using the aggregated master model, which at the same time may be deployed to the clients in the inference cluster to continue to produce inference results.

[0151] The operations in steps 8 to 10 may also be realized as described hereinafter in the Model Retraining section.

[0152] Steps 4 -10 may be repeated until the model has converged, i.e., the model is no longer changing significantly. But even in case of a total convergence (at least for some unknown period), Federated Learning is in general an iterative process as it is highly likely that there may be frequent / continuous local training at FL clients which may modify the model parameter aggregation results significantly or lead to a possible case of model/data drift (Step 7) which may result in a new cluster arrangement. Also, the loop may occur differently for each cluster and/or client, depending on updates of clients’ information, data patterns, computational and network resources, participation choices, and user requests.

[0153] Systems and methods are proposed herein for enabling federated learning deployment services. For example, the method described in Figure 4 may comprise receiving, from a user device, a message comprising deployment configuration information for a trained ML model, wherein the deployment configuration information comprises one or more policies for re-training the trained ML model. The method may further comprise sending, to a first group of client devices, a request to deploy the trained ML model at the first group of client devices and a request to receive responses of deployment information associated with the deploying the trained ML model. The method may further comprise receiving, from at least one of the first group of client devices, a response comprising the deployment information. The method may further comprise determining, based on the one or more policies for re-training the ML model and the received deployment information, to trigger re-training of the ML model. The method may further comprise sending, to a second group of client devices, a request to re-train the ML model.

[0154] The method described in Figure 4 may further comprise wherein the user device is a federated learning user, and wherein the client device is a federated learning client.

[0155] The method described in Figure 4 may further comprise wherein the policies for re-training the ML model comprise one or more trigger conditions for model re-training, and wherein the policies for re-training the ML model comprise performing, based on a met trigger condition, one or more actions to be taken by the ML model deployment service.

[0156] The method described in Figure 4 may further comprise wherein the one or more trigger conditions comprise at least one of detecting, by a first threshold number of the first group of client devices, data drift associated with the ML model or detecting, by a second threshold number of the first group of client devices, model drift associated with the ML model.

[0157] The method described in Figure 4 may further comprise wherein the one or more actions comprise adding a client device of the first group to the second group of client devices.

[0158] The method described in Figure 4 may further comprise wherein the deployment information comprises one or more of statistics of inferencing data, performance information of the deployed ML model, information associated with resource consumption for using the deployed ML model, or information associated with time consumption for using the deployed ML model.

[0159] The method described in Figure 4 may further comprise wherein the first group of client devices and the second group of client devices are determined based on a client clustering policy associated with the deployment configuration information.

For example, the system described in Figure 4 may comprise an apparatus comprising a processor and a memory, the apparatus further comprising computer-executable instructions stored in the memory of the apparatus which, when executed by the processor of the apparatus, cause the apparatus to perform operations. The operations may comprise receiving, from a service provider and at a first client device of a first group of client devices, a request to deploy a trained ML model at the first group of client devices and to send responses of deployment information associated with the deploying the trained ML model, wherein the request comprises one or more policies associated with re-training the ML model. The operations may further comprise sending, by the first client device, a response comprising the deployment information, wherein the service provider is configured to determine, based on the one or more policies for retraining the ML model and the sent deployment information, to trigger re-training of the ML model. The operations may further comprise receiving, by the first client device and a second group of client devices, wherein the first client device is associated with the second group of client device, a request to re-train the ML model.

[0160] The apparatus described in Figure 4 may further comprise wherein the first client device is a federated learning client.

[0161] The apparatus described in Figure 4 may further comprise wherein the policies for re-training the ML model comprise one or more trigger conditions for model re-training, and wherein the policies for re-training the ML model comprise performing, based on a met trigger condition, one or more actions to be taken to re-train the ML model.

[0162] The apparatus described in Figure 4 may further comprise wherein the one or more trigger conditions comprise at least one of detecting, by a first threshold number of the first group of client devices, data drift associated with the ML model or detecting, by a second threshold number of the first group of client devices, model drift associated with the ML model. [0163] The apparatus described in Figure 4 may further comprise wherein the one or more actions comprise adding a client device of the first group of client devices to the second group of client devices.

[0164] The apparatus described in Figure 4 may further comprise wherein the deployment information comprises one or more of: statistics of inferencing data, performance information of the deployed ML model, information associated with the resource consumption for using the deployed ML model, or information associated with time consumption for using the deployed ML model.

[0165] The apparatus described in Figure 4 may further comprise wherein the first group of client devices and the second group of client devices are determined based on a client clustering policy associated with the deployment configuration information.

[0166] FL Deployment Procedure Details. The following procedures provide more details or alternatives to the procedure shown in Figure 4:

[0167] 1) FL Model Discovery

[0168] Federated learning model deployment comprises the use of pretrained ML model(s) that FL clients participates in the training and/or inferencing of a model that the FL clients may have discovered from a FL deployment service or ML service. This section describes a FL client-initiated procedure to request the use of a ML model that may have been trained in a Federated Learning training process (or via other training method), which may be a complementary/alternative approach of the relevant procedures defined in Figure 4 (steps 1 to 3). The FL client may indicate that the client is able to participate as a federated learning client and provide information about data that may be used to train a ML model. The FL client may send a discovery request to a FL deployment service for a machine learning model that may be relevant to the dataset that the FL client has available. The FL deployment service may process the request to see if there are machine learning models that match the discovery requirements specified by the FL client and optionally may send a notification to FL users that may grant necessary permission for the FL client to receive the matched models and/or participate in the training process of the matched models. The FL deployment service may provide the matched model(s) to the FL client along with an associated policy that defines the metrics needed to be collected by the FL client (such as the information defined in “Client Reporting Instructions” in Table 2). The metrics capture statistical information about the data that the deployed model is applied to. The statistical information may comprise characteristics of the input data, the prediction results and the accuracy results, etc.

[0169] Figure 5 describes client-initiated model discovery

[0170] Step 1 : An FL client may issue a ML model discovery request to a FL deployment service to inquire about available ML models for training and/or inferencing. The discovery request may comprise: the FL client identifier, FL client type, ML application type, ML model requirements, client capabilities, client schedule and availability, available data and information about the data, and client willingness for training and/or inferencing (or other information about the client and/or the model as defined in Table 1 and Table 2). Steps 1-3 may be skipped if the FL client participated in the training of the model and has the model and associated monitoring policy (such as the information defined in “Client Reporting Instructions” in Table 2) locally.

[0171] Step 2: When the FL deployment service receives a model discovery request from a FL client, the FL deployment service may first send a notification to the FL user(s) and provide in the notification information of the FL client and discovery filter of the model (which is received in the request in step 1).

[0172] Step 3 : The FL user may return a response to the notification of whether to authorize the FL client access to the requested ML model. The response may also comprise a usage policy for when the FL client is allowed to use the ML model (e.g. “Model Usage Policy” in Table 2), what information the FL client needs to provide during usage of the ML model (e.g. ’’Client Reporting Instructions” in Table 2, information defined in Table 3), etc.

[0173] Step 4: When the FL deployment service determines that the FL client has authorization to retrieve a model, either through explicit approval from a FL user or a policy that the FL deployment service may access, the FL deployment service may provide the machine learning model and the associated policy to the FL client. The policy associated with the model may have the same information elements that are shown in Step 1.

[0174] If the client may participate in the training/inferencing of the machine learning model, the model discovery response may comprise the model usage policy, the performance characteristics that the model is expected to achieve, characteristics of the inference data that is to be monitored by the client when the model is deployed, etc. This policy may also comprise specification of the input data to the machine learning model that should be collected by the FL client, for possible use in future training. Table 3 shows examples of information that may be defined in the policy, which may be another embodiment of the parameters described in Table 2.

[0175] Table 3: Federated Learning Data Metrics Policy

[0176] Step 5: When the machine learning model is deployed to the FL client, the policy specified in Table 3 may indicate the data metrics about the input data, output data and ground truth that should be recorded or computed by FL client. The FL client also stores the new input data when the input data meets the criteria specified in the policy.

[0177] 2) FL Client Reporting

[0178] This section describes the procedure of FL client reporting which may be a complementary/alternative approach of the relevant procedures defined in Figure 4 (steps 4 to 6). The FL client may download the machine learning model and the associated deployment policy and the model may be deployed at the client. The downloaded policy describes the data collection process to be used to monitor the performance of the machine learning model on new data, comprising collecting statistics about the data (e.g. generate EDA results) and performance evaluations of the model. When the data collection requirements are met, the FL client may periodically report information about the collected data, without sharing the actual data. The FL deployment service may also send a request to the FL clients to retrieve the report information on demand as shown in Figure 5.

[0179] As depicted in Figure 6, the FL deployment service may aggregate these collected reports from some or all FL clients to determine when or if the ML model should be retrained. Alternatively, an FL client may determine that the model performance is not sufficient for the client needs and send a model retraining/update request or evaluation report.

[0180] The data collection reports may comprise statistics of the entire data set the client has or only the statistics of the new data collected that was not used in the training process for the model. Alternatively, the report may comprise statistics of both the new data and the complete data set. The report may further comprise information about the performance of the model, the resource or time consumption for using the model, etc.

[0181] Step 1 The FL user may send a message that specifies the conditions that indicate that the model should be retrained (or the model retraining policy as defined in Table 2). For example if the ground truth metrics indicate that the predictions are incorrect 5% of the time that may signal the FL deployment service to trigger a retraining process. Table 4 shows examples of information that may be provided by the FL user to the FL deployment service regarding model retraining conditions.

[0182] Table 4 - FL User Retraining Conditions [0183] Step 2 (optional): When a FL client is operating a machine learning model that has been trained (e.g. using the federated learning process), the client may not have the ability to retrain the model on its own because the client may not have access to all of the data that is used for training. This process comprises a message that the FL client may send to the FL deployment service to indicate that the FL client would like to initiate a retraining process. This request message may occur earlier in the call flows such that the FL deployment service may need to perform step 1 again to get refreshed reports from the other clients.

[0184] Step 3: The FL deployment service may send a message to all the FL clients to request a report of the information/data collected at the clients. Alternatively, the clients may have been configured to send this report autonomously based on specific events or periodic timing. The FL user may also initiate the process to request the client Report (not shown in the figure).

[0185] Step 4: When the FL client is required to Report to the FL deployment service, either through a configured event or an explicit request from the FL deployment service, the FL client prepares the report. The report may comprise Input data metrics, output data metrics, ground truth metrics and feature data storage metrics, as shown in Table 5.

[0186] Table 5: Example Information that may be provided in the client Report

[0187] Step 5: The FL clients may report information to the FL deployment service (similar to step 5 of Figure 4). Information shown in Table 5 may be sent to the FL deployment service in separate messages and separate intervals. Each of the clients may send a report to the FL deployment service, however some clients may not respond or may not respond in the timeframe needed by the FL deployment service (as shown in steps 5.1 and 5.2).

[0188] Step 6: When the FL deployment service receives the clients’ reports, the FL deployment service may analyze each message individually and/or in an aggregated manner. The FL deployment service may store the information from these messages in a manner that may be retrieved or accessed at a later time by referring to the Report ID, the Model ID, the FL client ID or combinations of these identifiers. [0189] When the FL deployment service analyzes the input data metrics of the Reports the FL deployment service may check to make sure that the input data distributions are similar to the input data of the training data used for the current model. In a federated learning deployment, however, each client may not have the same distribution of input data that is represented by combined model trained in a federated manner. The FL deployment service may compare the current input data metrics with previous input data metrics from the same client to determine if retraining is triggered/indicated.

[0190] When the FL deployment service analyzes the output data metrics of the Collected Data Reports the FL deployment service may check to make sure that the output data distributions are similar to the output data of the training data used for the current model. In a federated learning deployment, however, each client may not have the same distribution of output data that is represented by combined model trained in a federated manner. The FL deployment service may compare the current output data metrics with previous output data metrics from the same client to determine if retraining is triggered/indicated.

[0191] When the FL deployment service analyzes the ground truth metrics of the collected data reports the FL deployment service may check the correct predictions and the incorrect predictions to determine if the model performance is sufficient. The determination of the need to retrain is use case dependent and may be configured by a FL user.

[0192] Step 7: It is possible that some clients may not report to the FL deployment service successfully or in a timely manner. The FL deployment service may proceed with the aggregated analysis with only a subset of information from FL clients. The number of FL clients’ reports needed to proceed may be specified by the FL user in the Retraining Conditions message in step 1. Once the required reports are received by the FL deployment service, the FL deployment service may combine the information from each of the clients to compute a global effectiveness of the machine learning model deployed on the FL clients. This may indicate that the retraining process should be triggered/started.

[0193] Step 8: The FL deployment service may notify the FL user of the information reported from the FL clients.

[0194] 3) Model Retraining

[0195] The following section describes the procedure of triggering model retraining, which may be a complementary/altemative approach of the relevant procedures defined in Figure 4 (steps 8 to 10). If model retraining is signaled through a FL client request or the FL deployment service’s analysis of FL client reports, there may be an authorization request to a FL user before the retraining process is continued. The signal to retrain may be issued by the FL deployment service periodically or based on the analysis of the reports from the FL clients. The retraining configuration message from the FL deployment service to the FL clients may comprise a specification of a subset of data that should be used for retraining. The Model retraining request may optionally be sent to a subset of clients.

[0196] Figure 7 depicts a flowchart describing an example method of FL Model Retraining.

[0197] As the FL deployment service aggregates and analyzes the data from the clients participating in the FL Learning training process, the FL deployment service may determine that the aggregate of data from all of the clients has resulted in a dataset that is not balanced. A common practice in this case is to balance the dataset by removing over-represented data or artificially generating sets of the under-represented data. The procedure described in this section defines messaging to indicate to each client in a pool of clients which data the clients should use for training or retraining to achieve the statistics properties of the data set needed across of the clients.

[0198] Step 1 : The Fl Client retraining procedure begins with a signal to indicate that retraining needs to occur. This may be FL user initiated, FL Service initiated, or FL client initiated.

[0199] Step 2: The process may be configured to notify interested entities such as FL user or FL clients that the (re)training process is triggered/indicated.

[0200] Step 3 : The process may require an authorization to proceed with the (re)training process.

[0201] Step 4: Based on the aggregated statistics analysis completed in section 5.2, the FL deployment service may have determined, or have been provided the determination externally, such as from a FL user, the overall statistics of the set of data needed for the retraining and the clients that have these sets of data. For example, if there are 3 clients with an imbalanced set of data for 3 classes of images that the machine learning model is being trained to classify, the FL deployment service may configure the 3 clients such that the overall set of data is balanced. The process may involve configuring each client to provide subsets of data while excluding other data, or the process may comprise having a FL client down-sample the number of samples of a particular target classification. Additionally, the FL clients may be configured to use data from a specific period of time for the machine learning training. Table 6 shows examples of information that may be sent to the FL clients from the FL deployment service for configuring the retraining process.

[0202] Table 6 - FL Client Retraining Configuration

[0203] Step 5: The FL deployment service sends the retraining configuration message to each of the FL clients. The FL client may participate in federated learning processes to retrain/update the model.

[0204] The aforementioned management functionality to deploy FL/ML models to different client clusters may be realized in multiple ways. In the below subsections, embodiments for a RESTful API, a Publish/Sub scribe (Pub/Sub) API, and a GUI API are defined.

[0205] RESTful API

[0206] One of the approaches to realize the aforementioned FL deployment service functionality is through a RESTful API supported by the FL deployment service. The RESTful API may comprise one or more different types of resources hosted and stored by the FL deployment service which are accessed by FL clients and FL users via RESTful operations commonly referred to as methods (e g., PUT, POST, GET, DELETE). Each type of resource defined for the API may comprise a defined set of information elements commonly referred to as attributes. These resource types and their respective attributes provide a means for the FL deployment service to receive information from FL clients and FL users using RESTful protocols such as HTTP, process the information by performing FL deployment service specific tasks, and provide results of these operations back to the FL clients and FL users. Hence the RESTful APIs are used by FL clients and FL users to interface with the FL deployment service and access the supported capabilities of the FL deployment service in a RESTful manner.

[0207] The RESTful API of the FL deployment service may support one or more types of FL deployment service centric resources. These FL deployment resources may comprise attributes such as but not limited to those defined in the aforementioned tables and procedures. These attributes may be represented and stored within these resources using representation formats such as JSON and XML as well as others. The FL deployment service may support instantiating these resources upon receiving requests from FL users and FL clients.

[0208] Below are some examples of RESTful APIs that a FL deployment service may support:

[0209] Example 1 - Client Registration API

Endpoint: POST /clients/register

Payload:

{ "devicelD": "<unique_device_id>", "deviceSpecs": { "cpu": "<cpu_info>", "ram":

"<ram_info>", "network": "<network_info>" }, "dataCharacteristics": { "volume": "<data_volume>", "variety": "<data_variety>" } }

[0210] This API may be used for registering clients with their device specifications and data characteristics.

[0211] Example 2 - Cluster Creation API

Endpoint: POST /clusters/create

Response:

{

"clusteringMethod": "<cluster_method>", # ’user’, ‘policy’, or ‘ml_clustering’

“clusterType”: “<cluster_type>”, # ‘resource’, ‘data’ (ml)

“dataCharacteristicsID” : “<data_information>”,

“resourcelnformationlD” : “<resource_info_ID>”,

"clusterPolicy": "<cluster_policy_id>",

"mlClusterAlgorithm" : "<ml cluster algorithm>"

[0212] This API would be used for creating clusters of clients based on client compute and network resources, parent cluster, and FL cluster type ID.

[0213] Example 3 - Cluster Configuration API

Endpoint: POST /clusters/configure

Response:

"clusterType" : "<cluster_type>",

"parentcluster": "<parent_cluster_id>", "mlClusterTypeTD" : "<ml_cluster_type_id>"

[0214] This API would be used for configuring clusters based on their type, parent cluster, and FL cluster type ID.

[0215] Example 4 - Cluster Assignment API

Endpoint: GET /clusters/assign?deviceID=<unique_device_id>

Response: { "clusterlD": ["<assigned_cluster_id_l>", "<assigned_cluster_id_2>,...]"} [0216] This API would be used to assign clients to clusters based on their device specs and data characteristics. There may be multiple types of clusters classifications, such as task_cluster (for training vs. inference based on compute and network resources), ml_cluster (data heterogeneity related).

[0217] This API would be used by 'student' clients to make predictions using the trained models.

[0218] Example 5 - Cluster Reassignment API

Endpoint: PUT /clusters/reassign

Payload:

{ "devicelD": "<unique_device_id>", "deviceSpecs": { "cpu": "<updated_cpu_info>", "ram": "<updated_ram_info>", "network": "<updated_network_info>" }, "dataCharacteristics": { "volume": "<updated_data_volume>", "variety": "<updated_data_variety>" } } Response: { "clusterlD": "<new_assigned_cluster_id>" }

[0219] This API would be used to reassign clients to new clusters based on updated device specifications and data characteristics.

[0220] Example 6 - Model Update API

Endpoint: PUT /models/update

Payload: { "modellD": "<model_id>", "updatedModel": "<updated_model>" }

[0221] This API would be used by 'teacher' clients to update the models based on new training data or changes in the data distribution.

[0222] Publish/Subscribe (Pub/Sub) API

[0223] Another approach to realize the aforementioned FL deployment service functionality is through a Publish/Subscribe (Pub/Sub) API supported by the FL deployment service. The Pub/Sub API may comprise one or more different types of topics hosted and stored by the FL deployment service which are accessed by FL clients and FL users via publish and subscribe operations. Each type of topic defined for the API may comprise a defined set of information elements commonly referred to as attributes. These topics and their respective attributes may provide a means for the FL deployment service to receive information from FL clients and FL users using Pub/Sub protocols such as MQTT and AMQP, process the information by performing FL deployment service specific tasks, and provide results of these operations back to the FL clients and FL users. Hence the Pub/Sub APIs may be used by FL clients and FL users to interface with the FL deployment service and access the supported capabilities of the FL deployment service in a Pub/Sub manner.

[0224] The Pub/Sub API of the FL deployment service may support one or more types of FL deployment service centric topics. These FL deployment topics may comprise attributes such as but not limited to those defined in the aforementioned tables and procedures. These attributes may be represented and stored within these topics using representation formats such as JSON and XML as well as others. The FL deployment service may support instantiating these topics upon receiving requests from FL users and FL clients.

[0225] Below are some examples of Pub/Sub APIs that a FL deployment service may support:

[0226] Example 1 - Client Registration Topic:

[0227] Clients would publish their device specifications and data characteristics to a client registration topic. The payload might look something like this:

{

"devicelD": "<unique_device_id>",

"deviceSpecs": { "cpu": "<cpu_info>", "ram": "<ram_info>", "network": "<network_info>" },

"dataCharacteristics": { "volume": "<data_volume>", "variety": "<data_variety>" }

[0228] A server-side subscriber (the FL deployment service in this case) would listen to this topic, process the client registrations, and assign them to clusters.

[0229] Example 2 - Cluster Creation Topic: [0230] Users or server could publish cluster configurations to a cluster configuration topic. The message might look like this:

{

"clusterlD": "<cluster_id>",

"clusterType" : "<cluster_type>",

"parentcluster": "<parent_cluster_id>",

"mlClusterTypelD" : "<ml_cluster_type_id>"

[0231] Subscribers would listen to this topic to receive updates about cluster configurations.

[0232] Example 3 - Cluster Configuration Topic:

[0233] Users or the FL deployment service could publish cluster configurations to a cluster configuration topic. The message might look like this:

"clusterlD": "<cluster_id>",

"clusterType" : "<cluster_type>",

"parentcluster": "<parent_cluster_id>",

"mlClusterTypelD" : "<ml_cluster_type_id>"

}

[0234] Subscribers would listen to this topic to receive updates about cluster configurations.

[0235] Example 4 - Cluster Assignment Topic:

[0236] The server would publish cluster assignments to a cluster assignment topic. The message might look like this:

"devicelD": "<unique_device_id>",

"clusterlD" : "<assigned_cluster_id>"

[0237] Clients would subscribe to this topic to receive their cluster assignments.

[0238] Example 5 - Model Training Topic: [0239] ' Teacher¹ clients (clients in the training cluster) per each ML subcluster would publish trained models to a model training topic. The message might look like:

{

"clusterlD": "<cluster_id>",

"modelType": "<model_type>",

"trainedModel": "<trained_model>"

[0240] Other 'teacher¹ clients in the same cluster could subscribe to this topic to receive updates on the trained models.

[0241] Example 6 - Model Inference Topic:

[0242] 'Student' clients (clients in the inference cluster) per each ML subcluster would publish inference data to a model inference topic and receive predictions from 'teacher' clients. The message might look like:

"modellD": "<model_id>",

"inferenceData" : "<inference_data>"

[0243] ‘ Teacher' clients would subscribe to this topic to receive inference data, perform predictions, and publish the results.

[0244] Example 7 - Cluster Reassignment Topic:

[0245] Clients would publish updated device specifications and data characteristics to a cluster reassignment topic. The message may look like:

{

" devi celD " : " <uni que devi ce_i d> " ,

“clusterlD”: "<unique_cluster_id>",

"deviceSpecs": { "cpu": "<updated_cpu_info>", "ram": "<updated_ram_info>",

"network": "<updated_network_info>" },

"dataCharacteristics": { "volume": "<updated_data_volume>", "variety":

"<updated_data_variety>" } [0246] The FL deployment service and/or user may listen to this topic, process the updated client information, and reassign them to new clusters.

[0247] Example 8 - Model Update Topic:

[0248] ' Teacher¹ clients would publish updated models to a model update topic. The message may look like:

"modellD": "<model_id>",

"updatedModel" : "<updated_model>"

[0249] Other 'teacher¹ clients in the same cluster may subscribe to this topic to receive updates on the models.

[0250] GUI

[0251] Yet another approach to realize the aforementioned FL deployment service functionality is through a graphical user interface (GUI) centric API supported by the FL deployment service. The GUI API may comprise one or more different types of GUI features supported by the FL deployment service which are accessed by FL clients and FL users via GUI operations. The GUI API may comprise a defined set of information elements. The features and information elements may provide a means for the FL deployment service to receive information from FL clients and FL users using GUI operations, process the information by performing FL deployment service specific tasks, and provide results of these operations back to the FL clients and FL users. Hence, the GUI APIs may be used by FL clients and FL users to interface with the FL deployment service and access the supported capabilities of the FL deployment service in a GUI-based manner.

[0252] The GUI API of the FL deployment service may support one or more types of FL deployment service-centric features (e.g., panels, menus, canvas, etc.). These FL deployment features may comprise information elements such as but not limited to those defined in the aforementioned tables and procedures. These information elements may be represented using representation formats such as JSON and XML as well as others.

[0253] Below are some examples of GUI APIs that a FL deployment service may support:

[0254] Example 1 - Client Registration Panel: [0255] This panel allows clients to input and submit their device specifications and data characteristics. The panel may comprise form fields for CPU info, RAM info, network info, data volume, and data variety.

[0256] Example 2 - Client Cluster Creation Panel:

[0257] This panel allows users to input options to create client clusters based on clients’ compute and network resources and their data characteristics.

[0258] Example 3 - Cluster Configuration API

[0259] This dashboard displays options to configure client clusters. There may be multiple types of clusters classifications, such as task_cluster (for training vs. inference based on compute and network resources), ml cluster (data heterogeneity related). This panel allows users to input and submit cluster configurations. The panel may comprise form fields for cluster type, parent cluster, and FL cluster type ID. There may be a dropdown menu for selecting whether the cluster is for training or inference.

[0260] Example 4 - Cluster Assignment Dashboard:

[0261] This dashboard displays the cluster assignments of the clients. The dashboard may show a list or table of clients with their respective assigned clusters. A refresh button may be provided to update the dashboard as clients are reassigned to different clusters.

[0262] Example 5 - Cluster Reassignment Panel:

[0263] This panel allows clients to update their device specifications and data characteristics, triggering a reassignment to a new cluster. The panel may comprise similar form fields as the client registration panel, with a button to submit the updated information.

[0264] Example 6 - Model Update Module:

[0265] Similar to the model training module, this module may be accessible only to 'teacher' clients. The model training module allows the ‘teacher’ clients to upload updated models, which may be shared with 'student' clients.

[0266] Example 7 - Monitoring and Logging Panel:

[0267] This panel displays logs and metrics related to the system's operation. This may comprise cluster assignment updates, model training progress, inference results, etc.

[0268] Example 8 - Notifications Area:

[0269] This area displays notifications to the user about important events, such as successful model training, cluster reassignment, etc. [0270] Figure 11 A shows one embodiment of an example communications system 100 in which the methods and apparatuses described and claimed herein may be embodied. As shown, the example communications system 100 may comprise wireless transmit/receive units (WTRUs) 102a, 102b, 102c, 102d, 102e, 102f, and/or 102g (which generally or collectively may be referred to as WTRU 102), a radio access network (RAN) 103/104/105/103b/104b/l 05b, a core network 106/107/109, a public switched telephone network (PSTN) 108, the Internet 110, , other networks 112, and V2X server (or ProSe function and server) 113, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of the WTRUs 102a, 102b, 102c, 102d, 102e, 102f, 102g may be any type of apparatus or device configured to operate and/or communicate in a wireless environment. Although each WTRU 102a, 102b, 102c, 102d, 102e, 102f, 102g is depicted in Figures 11 A-l IE as a hand-held wireless communications apparatus, it is understood that with the wide variety of use cases contemplated for 5G wireless communications, each WTRU may comprise or be embodied in any type of apparatus or device configured to transmit and/or receive wireless signals, including, by way of example only, user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a tablet, a netbook, a notebook computer, a personal computer, a wireless sensor, consumer electronics, a wearable device such as a smart watch or smart clothing, a medical or eHealth device, a robot, industrial equipment, a drone, a vehicle such as a car, truck, train, or airplane, and the like.

[0271] The communications system 100 may also comprise a base station 114a and a base station 114b. Base stations 114a may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 110, and/or the other networks 112. Base stations 114b may be any type of device configured to wiredly and/or wirelessly interface with at least one of the RRHs (Remote Radio Heads) 118a, 118b, TRPs (Transmission and Reception Points) 119a, 119b, and/or RSUs (Roadside Units) 120a and 120b to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 110, the other networks 112, and/or V2X server (or ProSe function and server) 113. RRHs 118a, 118b may be any type of device configured to wirelessly interface with at least one of the WTRU 102c, to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 110, and/or the other networks 112. TRPs 119a, 119b may be any type of device configured to wirelessly interface with at least one of the WTRU 102d, to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 110, and/or the other networks 112. RSUs 120a and 120b may be any type of device configured to wirelessly interface with at least one of the WTRU 102e or 102f, to facilitate access to one or more communication networks, such as the core network 106/107/109, the Internet 110, the other networks 112, and/or V2X server (or ProSe function and server) 113. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may comprise any number of interconnected base stations and/or network elements.

[0272] The base station 114a may be part of the RAN 103/104/105, which may also comprise other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114b may be part of the RAN 103b/l 04b/l 05b, which may also comprise other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a may be configured to transmit and/or receive wireless signals within a particular geographic region, which may be referred to as a cell (not shown). The base station 114b may be configured to transmit and/or receive wired and/or wireless signals within a particular geographic region, which may be referred to as a cell (not shown). The cell may further be divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in an embodiment, the base station 114a may comprise three transceivers, e.g., one for each sector of the cell. In an embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and, therefore, may utilize multiple transceivers for each sector of the cell.

[0273] The base stations 114a may communicate with one or more of the WTRUs 102a, 102b, 102c over an air interface 115/116/117, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, cmWave, mmWave, etc.). The air interface 115/116/117 may be established using any suitable radio access technology (RAT). [0274] The base stations 114b may communicate with one or more of the RRHs 1 18a, 118b, TRPs 119a, 119b, and/or RSUs 120a and 120b, over a wired or air interface 115b/l 16b/l 17b, which may be any suitable wired (e.g., cable, optical fiber, etc.) or wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, cmWave, mmWave, etc.). The air interface 115b/l 16b/l 17b may be established using any suitable radio access technology (RAT).

[0275] The RRHs 118a, 118b, TRPs 119a, 119b and/or RSUs 120a, 120b, may communicate with one or more of the WTRUs 102c, 102d, 102e, 102f over an air interface 115c/l 16c/l 17c, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, cmWave, mmWave, etc.). The air interface 115c/l 16c/l 17c may be established using any suitable radio access technology (RAT).

[0276] The WTRUs 102a, 102b, 102c, 102d, 102e, 102f, and/or 102g may communicate with one another over an air interface 115d/l 16d/l 17d (not shown in the figures), which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, cmWave, mmWave, etc.). The air interface 115d/l 16d/l 17d may be established using any suitable radio access technology (RAT).

[0277] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 103/104/105 and the WTRUs 102a, 102b, 102c, or RRHs 118a, 118b, TRPs 119a, 119b and RSUs 120a, 120b, in the RAN 103b/l 04b/l 05b and the WTRUs 102c, 102d, 102e, 102f, may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115/116/117 or 115c/l 16c/l 17c respectively using wideband CDMA (WCDMA). WCDMA may comprise communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may comprise High-Speed Downlink Packet Access (HSDPA) and/or High- Speed Uplink Packet Access (HSUPA).

[0278] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c, or RRHs 118a, 118b, TRPs 119a, 119b, and/or RSUs 120a, 120b, in the RAN 103b/l 04b/l 05b and the WTRUs 102c, 102d, may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 115/116/117 or 115c/l 16c/l 17c respectively using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A). Tn the future, the air interface 115/116/117 may implement 3GPP NR technology. The LTE and LTE-A technology comprises LTE D2D and V2X technologies and interface (such as Sidelink communications, etc.) The 3GPP NR technology comprises NR V2X technologies and interface (such as Sidelink communications, etc.)

[0279] In an embodiment, the base station 114a in the RAN 103/104/105 and the WTRUs 102a, 102b, 102c, or RRHs 118a, 118b, TRPs 119a, 119b and/or RSUs 120a, 120b, in the RAN 103b/l 04b/l 05b and the WTRUs 102c, 102d, 102e, 102f may implement radio technologies such as IEEE 802.16 (e.g., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 IX, CDMA2000 EV-DO, Interim Standard 2000 (IS- 2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

[0280] The base station 114c in Figure 11 A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, and the like. In an embodiment, the base station 114c and the WTRUs 102e, may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114c and the WTRUs 102d, may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114c and the WTRUs 102e, may utilize a cellularbased RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.) to establish a picocell or femtocell. As illustrated in Figure 11 A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114c may not be required to access the Internet 110 via the core network 106/107/109.

[0281] The RAN 103/104/105 and/or RAN 103b/104b/105b may be in communication with the core network 106/107/109, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. For example, the core network 106/107/109 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication.

[0282] Although not shown in Figure 11 A, it will be appreciated that the RAN 103/104/105 and/or RAN 103b/104b/105b and/or the core network 106/107/109 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 103/104/105 and/or RAN 103b/l 04b/ 105b or a different RAT. For example, in addition to being connected to the RAN 103/104/105 and/or RAN 103b/104b/105b, which may be utilizing an E-UTRA radio technology, the core network 106/107/109 may also be in communication with another RAN (not shown) employing a GSM radio technology.

[0283] The core network 106/107/109 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d, 102e to access the PSTN 108, the Internet 110, and/or other networks 112. The PSTN 108 may comprise circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet 110 may comprise a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and the internet protocol (IP) in the TCP/IP internet protocol suite. The networks 112 may comprise wired or wireless communications networks owned and/or operated by other service providers. For example, the networks 112 may comprise another core network connected to one or more RANs, which may employ the same RAT as the RAN 103/104/105 and/or RAN 103b/104b/105b or a different RAT.

[0284] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may comprise multi-mode capabilities, e.g., the WTRUs 102a, 102b, 102c, 102d, and 102e may comprise multiple transceivers for communicating with different wireless networks over different wireless links. For example, the WTRU 102e shown in Figure 11 A may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114c, which may employ an IEEE 802 radio technology.

[0285] Figure 1 IB is a block diagram of an example apparatus or device configured for wireless communications in accordance with the embodiments shown herein, such as for example, a WTRU 102. As shown in Figure 1 IB, the example WTRU 102 may comprise a processor 118, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 113, a display/touchpad/indicators 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and other peripherals 138. It will be appreciated that the WTRU 102 may comprise any sub-combination of the foregoing elements while remaining consistent with an embodiment. Also, embodiments contemplate that the base stations 114a and 114b, and/or the nodes that base stations 114a and 114b may represent, such as but not limited to transceiver station (BTS), a Node-B, a site controller, an access point (AP), a home node-B, an evolved home node-B (eNodeB), a home evolved node-B (HeNB), a home evolved node-B gateway, and proxy nodes, among others, may comprise some or all of the elements depicted in Figure 1 IB and described herein.

[0286] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit/receive element 122. While Figure 1 IB depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0287] The transmit/receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface 115/116/117. For example, in an embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In an embodiment, the transmit/receive element 122 may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet an embodiment, the transmit/receive element 122 may be configured to transmit and receive both RF and light signals. It will be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

[0288] In addition, although the transmit/receive element 122 is depicted in Figure 1 IB as a single element, the WTRU 102 may comprise any number of transmit/receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in an embodiment, the WTRU 102 may comprise two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 115/116/117.

[0289] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit/receive element 122 and to demodulate the signals that are received by the transmit/receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may comprise multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as UTRA and IEEE 802.11, for example.

[0290] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker/mi crophone 124, the keypad 126, and/or the display/touchpad/indicators 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker/microphone 124, the keypad 126, and/or the display/touchpad/indicators 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and/or the removable memory 132. The nonremovable memory 130 may comprise random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may comprise a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In an embodiment, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0291] The processor 118 may receive power from the power source 134, and may be configured to distribute and/or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may comprise one or more dry cell batteries, solar cells, fuel cells, and the like.

[0292] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 115/116/117 from a base station (e.g., base stations 114a, 114b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.

[0293] The processor 118 may further be coupled to other peripherals 138, which may comprise one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals 138 may comprise various sensors such as an accelerometer, biometrics (e.g., finger print) sensors, an e- compass, a satellite transceiver, a digital camera (for photographs or video), a universal serial bus (USB) port or other interconnect interfaces, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, and the like.

[0294] The WTRU 102 may be embodied in other apparatuses or devices, such as a sensor, consumer electronics, a wearable device such as a smart watch or smart clothing, a medical or eHealth device, a robot, industrial equipment, a drone, a vehicle such as a car, truck, train, or airplane. The WTRU 102 may connect to other components, modules, or systems of such apparatuses or devices via one or more interconnect interfaces, such as an interconnect interface that may comprise one of the peripherals 138.

[0295] Figure 11C is a system diagram of the RAN 103 and the core network 106 according to an embodiment. As noted above, the RAN 103 may employ a UTRA radio technology to communicate with the WTRUs 102a, 102b, and 102c over the air interface 115. The RAN 103 may also be in communication with the core network 106. As shown in Figure 11C, the RAN 103 may comprise Node-Bs 140a, 140b, 140c, which may each comprise one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 115. The Node-Bs 140a, 140b, 140c may each be associated with a particular cell (not shown) within the RAN 103. The RAN 103 may also comprise RNCs 142a, 142b. It will be appreciated that the RAN 103 may comprise any number of Node-Bs and RNCs while remaining consistent with an embodiment.

[0296] As shown in Figure 11C, the Node-Bs 140a, 140b may be in communication with the RNC 142a. Additionally, the Node-B 140c may be in communication with the RNC 142b. The Node-Bs 140a, 140b, 140c may communicate with the respective RNCs 142a, 142b via an lub interface. The RNCs 142a, 142b may be in communication with one another via an lur interface. Each of the RNCs 142a, 142b may be configured to control the respective Node-Bs 140a, 140b, 140c to which it is connected. In addition, each of the RNCs 142a, 142b may be configured to carry out or support other functionality, such as outer loop power control, load control, admission control, packet scheduling, handover control, macro-diversity, security functions, data encryption, and the like.

[0297] The core network 106 shown in Figure 11C may comprise a media gateway (MGW) 144, a mobile switching center (MSC) 146, a serving GPRS support node (SGSN) 148, and/or a gateway GPRS support node (GGSN) 150. While each of the foregoing elements are depicted as part of the core network 106, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0298] The RNC 142a in the RAN 103 may be connected to the MSC 146 in the core network 106 via an luCS interface. The MSC 146 may be connected to the MGW 144. The MSC 146 and the MGW 144 may provide the WTRUs 102a, 102b, 102c with access to circuit- switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices.

[0299] The RNC 142a in the RAN 103 may also be connected to the SGSN 148 in the core network 106 via an luPS interface. The SGSN 148 may be connected to the GGSN 150. The SGSN 148 and the GGSN 150 may provide the WTRUs 102a, 102b, 102c with access to packet- switched networks, such as the Internet 110, to facilitate communications between and the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0300] As noted above, the core network 106 may also be connected to the networks 112, which may comprise other wired or wireless networks that are owned and/or operated by other service providers.

[0301] Figure 1 ID is a system diagram of the RAN 104 and the core network 107 according to an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, and 102c over the air interface 116. The RAN 104 may also be in communication with the core network 107.

[0302] The RAN 104 may comprise eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may comprise any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each comprise one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In an embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

[0303] Each of the eNode-Bs 160a, 160b, and 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink and/or downlink, and the like. As shown in Figure HD, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface.

[0304] The core network 107 shown in Figure 1 ID may comprise a mobility management gateway (MME) 162, a serving gateway 164, and a packet data network (PDN) gateway 166. While each of the foregoing elements are depicted as part of the core network 107, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0305] The MME 162 may be connected to each of the eNode-Bs 160a, 160b, and 160c in the RAN 104 via an SI interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation/deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.

[0306] The serving gateway 164 may be connected to each of the eNode-Bs 160a, 160b, and 160c in the RAN 104 via the SI interface. The serving gateway 164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102c. The serving gateway 164 may also perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like.

[0307] The serving gateway 164 may also be connected to the PDN gateway 166, which may provide the WTRUs 102a, 102b, 102c with access to packet- switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0308] The core network 107 may facilitate communications with other networks. For example, the core network 107 may provide the WTRUs 102a, 102b, 102c with access to circuit- switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the core network 107 may comprise, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the core network 107 and the PSTN 108. In addition, the core network 107 may provide the WTRUs 102a, 102b, 102c with access to the networks 112, which may comprise other wired or wireless networks that are owned and/or operated by other service providers.

[0309] Figure 1 IE is a system diagram of the RAN 105 and the core network 109 according to an embodiment. The RAN 105 may be an access service network (ASN) that employs IEEE 802. 16 radio technology to communicate with the WTRUs 102a, 102b, and 102c over the air interface 117. As will be further discussed below, the communication links between the different functional entities of the WTRUs 102a, 102b, 102c, the RAN 105, and the core network 109 may be defined as reference points.

[0310] As shown in Figure HE, the RAN 105 may comprise base stations 180a, 180b, 180c, and an ASN gateway 182, though it will be appreciated that the RAN 105 may comprise any number of base stations and ASN gateways while remaining consistent with an embodiment. The base stations 180a, 180b, 180c may each be associated with a particular cell in the RAN 105 and may comprise one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 117. In an embodiment, the base stations 180a, 180b, 180c may implement MIMO technology. Thus, the base station 180a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a. The base stations 180a, 180b, 180c may also provide mobility management functions, such as handoff triggering, tunnel establishment, radio resource management, traffic classification, quality of service (QoS) policy enforcement, and the like. The ASN gateway 182 may serve as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network 109, and the like.

[0311] The air interface 117 between the WTRUs 102a, 102b, 102c and the RAN 105 may be defined as an R1 reference point that implements the IEEE 802.16 specification. In addition, each of the WTRUs 102a, 102b, and 102c may establish a logical interface (not shown) with the core network 109. The logical interface between the WTRUs 102a, 102b, 102c and the core network 109 may be defined as an R2 reference point, which may be used for authentication, authorization, IP host configuration management, and/or mobility management.

[0312] The communication link between each of the base stations 180a, 180b, and 180c may be defined as an R8 reference point that comprises protocols for facilitating WTRU handovers and the transfer of data between base stations. The communication link between the base stations 180a, 180b, 180c and the ASN gateway 182 may be defined as an R6 reference point. The R6 reference point may comprise protocols for facilitating mobility management based on mobility events associated with each of the WTRUs 102a, 102b, 102c.

[0313] As shown in Figure 1 IE, the RAN 105 may be connected to the core network 109. The communication link between the RAN 105 and the core network 109 may defined as an R3 reference point that comprises protocols for facilitating data transfer and mobility management capabilities, for example. The core network 109 may comprise a mobile IP home agent (MIP-HA) 184, an authentication, authorization, accounting (AAA) server 186, and a gateway 188. While each of the foregoing elements are depicted as part of the core network 109, it will be appreciated that any one of these elements may be owned and/or operated by an entity other than the core network operator.

[0314] The MIP-HA may be responsible for IP address management, and may enable the WTRUs 102a, 102b, and 102c to roam between different ASNs and/or different core networks. The MIP-HA 184 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The AAA server 186 may be responsible for user authentication and for supporting user services. The gateway 188 may facilitate interworking with other networks. For example, the gateway 188 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. In addition, the gateway 188 may provide the WTRUs 102a, 102b, 102c with access to the networks 112, which may comprise other wired or wireless networks that are owned and/or operated by other service providers.

[0315] Although not shown in Figure 1 IE, it will be appreciated that the RAN 105 may be connected to other ASNs and the core network 109 may be connected to other core networks. The communication link between the RAN 105 the other ASNs may be defined as an R4 reference point, which may comprise protocols for coordinating the mobility of the WTRUs 102a, 102b, 102c between the RAN 105 and the other ASNs. The communication link between the core network 109 and the other core networks may be defined as an R5 reference, which may comprise protocols for facilitating interworking between home core networks and visited core networks.

[0316] The core network entities described herein and shown in Figures 11 A, 11C, 1 ID, and 1 IE are identified by the names given to those entities in certain existing 3GPP specifications, but it is understood that in the future those entities and functionalities may be identified by other names and certain entities or functions may be combined in future specifications published by 3GPP, including future 3GPP NR specifications. Thus, the particular network entities and functionalities described and shown in Figures 11 A, 1 IB, 11C, 1 ID, and 1 IE are provided by way of example only, and it is understood that the subject matter disclosed and claimed herein may be embodied or implemented in any similar communication system, whether presently defined or defined in the future.

[0317] Figure 1 IF is a block diagram of an exemplary computing system 90 in which one or more apparatuses of the communications networks shown in Figures 11A, 11C, 1 ID and 1 IE may be embodied, such as certain nodes or functional entities in the RAN 103/104/105, Core Network 106/107/109, PSTN 108, Internet 110, or Other Networks 112. Computing system 90 may comprise a computer or server and may be controlled primarily by computer readable instructions, which may be in the form of software, wherever, or by whatever means such software is stored or accessed. Such computer readable instructions may be executed within a processor 91, to cause computing system 90 to do work. The processor 91 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor 91 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the computing system 90 to operate in a communications network. Coprocessor 81 is an optional processor, distinct from main processor 91, that may perform additional functions or assist processor 91. Processor 91 and/or coprocessor 81 may receive, generate, and process data related to the methods and apparatuses disclosed herein.

[0318] In operation, processor 91 fetches, decodes, and executes instructions, and transfers information to and from other resources via the computing system’s main data-transfer path, system bus 80. Such a system bus connects the components in computing system 90 and defines the medium for data exchange. System bus 80 typically comprises data lines for sending data, address lines for sending addresses, and control lines for sending interrupts and for operating the system bus. An example of such a system bus 80 is the PCI (Peripheral Component Interconnect) bus.

[0319] Memories coupled to system bus 80 comprise random access memory (RAM) 82 and read only memory (ROM) 93. Such memories comprise circuitry that allows information to be stored and retrieved. ROMs 93 generally comprise stored data that cannot easily be modified. Data stored in RAM 82 may be read or changed by processor 91 or other hardware devices. Access to RAM 82 and/or ROM 93 may be controlled by memory controller 92. Memory controller 92 may provide an address translation function that translates virtual addresses into physical addresses as instructions are executed. Memory controller 92 may also provide a memory protection function that isolates processes within the system and isolates system processes from user processes. Thus, a program running in a first mode may access only memory mapped by its own process virtual address space; it cannot access memory within another process’ s virtual address space unless memory sharing between the processes has been set up.

[0320] In addition, computing system 90 may comprise peripherals controller 83 responsible for communicating instructions from processor 91 to peripherals, such as printer 94, keyboard 84, mouse 95, and disk drive 85.

[0321] Display 86, which is controlled by display controller 96, is used to display visual output generated by computing system 90. Such visual output may comprise text, graphics, animated graphics, and video. The visual output may be provided in the form of a graphical user interface (GUI). Display 86 may be implemented with a CRT-based video display, an LCDbased flat-panel display, gas plasma-based flat-panel display, or a touch-panel. Display controller 96 includes electronic components required to generate a video signal that is sent to display 86.

[0322] Further, computing system 90 may comprise communication circuitry, such as for example a network adapter 97, that may be used to connect computing system 90 to an external communi cations network, such as the RAN 103/104/105, Core Network 106/107/109, PSTN 108, Internet 110, or Other Networks 112 of Figures 11 A, 1 IB, 11C, 1 ID, and 1 IE, to enable the computing system 90 to communicate with other nodes or functional entities of those networks. The communication circuitry, alone or in combination with the processor 91, may be used to perform the transmitting and receiving steps of certain apparatuses, nodes, or functional entities described herein.

[0323] Figure 11G shows one embodiment of an example communications system 111 in which the methods and apparatuses described and claimed herein may be embodied. As shown, the example communications system 111 may comprise wireless transmit/receive units (WTRUs) A, B, C, D, E, F, a base station, a V2X server, and a RSUs A and B, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. One or several or all WTRUs A, B, C, D, E can be out of range of the network (for example, in the figure out of the cell coverage boundary shown as the dash line). WTRUs A, B, C form a V2X group, among which WTRU A is the group lead and WTRUs B and C are group members. WTRUs A, B, C, D, E, F may communicate over Uu interface or Sidelink (PC5) interface.

[0324] It is understood that any or all of the apparatuses, systems, methods and processes described herein may be embodied in the form of computer executable instructions (e.g., program code) stored on a computer-readable storage medium which instructions, when executed by a processor, such as processors 118 or 91, cause the processor to perform and/or implement the systems, methods and processes described herein. Specifically, any of the steps, operations or functions described herein may be implemented in the form of such computer executable instructions, executing on the processor of an apparatus or computing system configured for wireless and/or wired network communications. Computer readable storage media comprise volatile and nonvolatile, removable and non-removable media implemented in any non-transitory (e.g., tangible or physical) method or technology for storage of information, but such computer readable storage media do not comprise signals. Computer readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD- ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other tangible or physical medium which may be used to store the desired information and which may be accessed by a computing system.

Claims

CLAIMS What is claimed is:

1. A method performed by a Machine Learning (ML) model deployment service comprising: receiving, from a user device, a message comprising deployment configuration information for a trained ML model, wherein the deployment configuration information comprises one or more policies for re-training the trained ML model; sending, to a first group of client devices, a request to deploy the trained ML model at the first group of client devices and a request to receive responses of deployment information associated with the deploying the trained ML model; receiving, from at least one of the first group of client devices, a response comprising the deployment information; determining, based on the one or more policies for re-training the ML model and the received deployment information, to trigger re-training of the ML model; and sending, to a second group of client devices, a request to re-train the ML model.

2. The method of claim 1, wherein the user device is a federated learning user, and wherein the client device is a federated learning client.

3. The method of claim 1, wherein the policies for re-training the ML model comprise one or more trigger conditions for model re-training, and wherein the policies for re-training the ML model comprise performing, based on a met trigger condition, one or more actions to be taken by the ML model deployment service.

4. The method of claim 3, wherein the one or more trigger conditions comprise at least one of detecting, by a first threshold number of the first group of client devices, data drift associated with the ML model or detecting, by a second threshold number of the first group of client devices, model drift associated with the ML model.

5. The method of claim 3, wherein the one or more actions comprise adding a client device of the first group to the second group of client devices.

6. The method of claim 1, wherein the deployment information comprises one or more of: statistics of inferencing data, performance information of the deployed ML model, information associated with resource consumption for using the deployed ML model, or information associated with time consumption for using the deployed ML model.

7. The method of claim 1, wherein the first group of client devices and the second group of client devices are determined based on a client clustering policy associated with the deployment configuration information.

8. An apparatus comprising a processor and a memory, the apparatus further comprising computer-executable instructions stored in the memory of the apparatus which, when executed by the processor of the apparatus, cause the apparatus to perform operations comprising: receiving, from a service provider and at a first client device of a first group of client devices, a request to deploy a trained ML model at the first group of client devices and to send responses of deployment information associated with the deploying the trained ML model, wherein the request comprises one or more policies associated with re-training the ML model; sending, by the first client device, a response comprising the deployment information, wherein the service provider is configured to determine, based on the one or more policies for retraining the ML model and the sent deployment information, to trigger re-training of the ML model; and receiving, by the first client device and a second group of client devices, wherein the first client device is associated with the second group of client device, a request to re-train the ML model.

9. The apparatus of claim 8, wherein the first client device is a federated learning client.

10. The apparatus of claim 8, wherein the policies for re-training the ML model comprise one or more trigger conditions for model re-training, and wherein the policies for re-training the ML model comprise performing, based on a met trigger condition, one or more actions to be taken to re-train the ML model.

11. The apparatus of claim 10, wherein the one or more trigger conditions comprise at least one of detecting, by a first threshold number of the first group of client devices, data drift associated with the ML model or detecting, by a second threshold number of the first group of client devices, model drift associated with the ML model.

12. The apparatus of claim 10, wherein the one or more actions comprise adding a client device of the first group of client devices to the second group of client devices.

13. The apparatus of claim 8, wherein the deployment information comprises one or more of: statistics of inferencing data, performance information of the deployed ML model, information associated with the resource consumption for using the deployed ML model, or information associated with time consumption for using the deployed ML model.

14. The apparatus of claim 8, wherein the first group of client devices and the second group of client devices are determined based on a client clustering policy associated with the deployment configuration information.