WO2023037319A1 - Real-time function-as-a-service system - Google Patents

Real-time function-as-a-service system Download PDF

Info

Publication number
WO2023037319A1
WO2023037319A1 PCT/IB2022/058527 IB2022058527W WO2023037319A1 WO 2023037319 A1 WO2023037319 A1 WO 2023037319A1 IB 2022058527 W IB2022058527 W IB 2022058527W WO 2023037319 A1 WO2023037319 A1 WO 2023037319A1
Authority
WO
WIPO (PCT)
Prior art keywords
cloud
multinode
network node
resource
based system
Prior art date
Application number
PCT/IB2022/058527
Other languages
French (fr)
Inventor
Mark Peter SZALAY
László Toka
Péter MÁTRAY
Original Assignee
Telefonaktiebolaget Lm Ericsson (Publ)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget Lm Ericsson (Publ) filed Critical Telefonaktiebolaget Lm Ericsson (Publ)
Publication of WO2023037319A1 publication Critical patent/WO2023037319A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network

Definitions

  • the present disclosure is related to wireless communication systems and more particularly to a real-time function-as-a-service system.
  • FIG. 1 illustrates an example of a new radio (“NR”) network (e.g., a 5th Generation (“5G”) network) including a 5G core (“5GC”) network 130, network nodes 120a-b (e.g., 5G base station (“gNB”)), multiple communication devices 1 10 (also referred to as user equipment (“UE”)).
  • NR new radio
  • 5G 5th Generation
  • 5GC 5G core
  • gNB 5G base station
  • UE user equipment
  • FaaS Function-as-a-Service
  • 5G internet of things
  • RT real-time
  • a method performed by a network node of a multinode cloud-based system includes receiving a deployment request for an application function (“AF”).
  • the method further includes determining requirements of the AF including a latency threshold associated with the AF.
  • the method further includes determining information associated with the multinode cloud-based system.
  • the method further includes determining a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloudbased system.
  • a network node, computer program, computer program product, or non-transitory computer-readable medium is provided to perform the above method.
  • Certain embodiments may provide one or more of the following technical advantages.
  • the proposed cloud design allows for new kinds of use-cases which the current cloud design is unable to implement:
  • Such a usecase could be the remote-controlled factories.
  • Novel applications of Industry 4.0 require reliable wireless connections, ultra-low end-to-end latency, high data rates, and supporting a massive number of devices. If the cloud system guarantees these, there is may be no need to run time-critical functions locally, i.e., it is not necessary to have your own hardware. Opening cloud and edge computing infrastructures for time-critical applications could be an appropriate solution for e.g., cloud-assisted safety systems, tactile internet, virtual and augmented reality.
  • a Latency Quality of Service (“QoS”) guarantees for users is provided.
  • Current cloud systems are not able to guarantee the worst-case execution time of the user demanded function.
  • the proposed RT-FaaS can guarantee the maximal return time which is essential for hard real-time applications.
  • simultaneous execution of RT and non-RT functions can be performed.
  • the proposed RT-FaaS assume a multinode multiprocessor cloud computing system, i.e., a cluster of servers connected by a LAN, and managed by the RT-FaaS platform with the goal of scheduling the user demanded functions such that the RT tasks are assigned to a minimum number of processors of the server cluster which are able to guarantee the deadline requirements are met. This operation leaves as many processors as possible to process the remaining non-RT functions, thus, ensures that they will not starve for the CPU cycle.
  • FIG. 1 is a schematic diagram illustrating an example of a 5 th generation (“5G”) network
  • FIG. 2 is a block diagram illustrating an example of a system architecture according to some embodiments
  • FIG. 3 is a block diagram illustrating an example of components of a server managed by RT-FaaS according to some embodiments
  • FIG. 4 is a flow chart illustrating an example of a partitioning procedure according to some embodiments.
  • FIG. 5 is a block diagram illustrating an example of a distributed system architecture according to some embodiments.
  • FIG. 6 is a flow chart illustrating an example of operations of a network node according to some embodiments of inventive concepts
  • FIG. 7 is a block diagram of a communication system in accordance with some embodiments.
  • FIG. 8 is a block diagram of a user equipment in accordance with some embodiments
  • FIG. 9 is a block diagram of a network node in accordance with some embodiments.
  • FIG. 10 is a block diagram of a host computer communicating with a user equipment in accordance with some embodiments.
  • FIG. 11 is a block diagram of a virtualization environment in accordance with some embodiments.
  • FIG. 12 is a block diagram of a host computer communicating via a base station with a user equipment over a partially wireless connection in accordance with some embodiments in accordance with some embodiments.
  • RT real-time
  • cloud-based networks can be challenging.
  • the real-time infrastructure must ensure deterministic communication and execution of the applications.
  • Examples of terminology and rules of RT applications include that “the high percentile delay between two application components must be less than x (bounded latencies)” or that “the worst-case execution time (“WCET”) of the function must be y”.
  • WCET worst-case execution time
  • the following must be predetermined: i) the specific hardware lying in the architecture and communicating to each other, ii) the number of them, iii) the network among them, iv) supported protocols, v) the operating systems (“OS”) of the servers, vi) the memory size reserving to the network packets, and vii) the application characteristic.
  • the characteristic defines whether it is an aperiodic, periodic, or sporadic task, its input, and output data rate, its execution depending on the input data or not, how much data source it has.
  • the essential characteristic of any cloud system is dynamism. It manages a heterogeneous system (e.g., multiple types of hardware can be included) and manages a continuously changing number of nodes. Geographically distributed servers can form an edge-cloud infrastructure.
  • the user deployed applications have dynamic characteristics such as i) the dynamic number of deployed applications (during runtime applications can be deployed, deleted, or updated anytime), ii) the number of external data sources are changeable, iii) dynamically varying input and output rate and quantity, iv) application execution may depend on the input data, v) various application lifetime in function as a service (“FaaS”) and vi) whether functions are communicating to other applications or services provided by the FaaS.
  • FaaS application lifetime in function as a service
  • FaaS real-time FaaS
  • RT- FaaS real-time FaaS
  • Faas platforms are described below. Thanks to its flexibility benefits, serverless computing can be effective in the field of cloud computing. It can allow developers to focus only on application development and not care about the server management and auxiliary functions by providing resource allocation, fast response time, schedulability, scalability, resiliency, and the deployed applications' upgradeability. Function-as-a-Service makes it possible for users to run their microservice applications without any management effort. Most of the FaaS platforms rely on container technologies, like LXC or Docker, and container orchestration systems such as Kubernetes. Typically, the users' source code is wrapped into containers that are scheduled and deployed by Kubernetes. Containerization provides isolation of processes and libraries with low-performance overhead.
  • Real-time physical networks are described below.
  • the underlying network of an RT-FaaS system can be highly reliable and provide bounded latency and low jitter.
  • the two networking standards are IEEE 802.1 TSN and IETF DetNet, which provide deterministic and reliable packet forwarding for link-layer and network-layer services.
  • Real-time virtual (within host) networks are described below.
  • a Linux communication stack can be meant for real-time robotic applications.
  • the Linux kernel - which is not optimized for bounded latencies - greatly enhances the deterministic nature of communications if there is no significant non-critical concurrent traffic in the system. If there is, it can disrupt the bounded latencies.
  • network stacks with the specific purpose of providing deterministic networking.
  • an open-source hard real-time network protocol stack for Linux equipped with real-time extension RTAI or Xenomai can be provided.
  • UDP/IP, TCP/IP, ICMP, and ARP can be implemented in a deterministic way and RT and non-RT traffic can be handled jointly.
  • Data flow graphs are popular program representations within a single node environment where graph nodes correspond to processing primitives and graph edges describe control flow. To improve CPU cache locality and exploit data- level parallelism, nodes usually process data in batches.
  • a scheduler for run-to- completion packet processing engines can be provided and can use controlled queuing to efficiently reconstruct fragmented batches in accordance with strict service-level objectives (“SLOs”).
  • SLOs strict service-level objectives
  • Xen hypervisor handles real-time and non-real-time traffic through its common interface, but includes limitations that could result in long or unpredictable network traffic latencies between virtual machines (“VMs”).
  • VMs virtual machines
  • a Virtualization-Aware Traffic Control framework can improve predictability and reduce the delay for latency-sensitive applications.
  • Real-time containers are described below.
  • critical containers can cohabit with traditional non-real-time containers on the same hardware - on top of a Linux kernel patched with a hard real-time co-kernel.
  • Preemptive fixed-priority scheduling can be selected as the underlying task scheduling.
  • a real-time deadline-based scheduling policy can be built into the Linux kernel to provide temporal scheduling guarantees to different co-located containers.
  • a hierarchical scheduling system that is compatible with the most used containerbased virtualization solutions and extended the SCHED DEADLINE scheduling policy can be provided.
  • the multiprocessor scheduling approaches can be divided into two categories: partitioned and global techniques.
  • the former uses a partitioning method to spread the tasks onto processors and runs a mono-processor scheduling algorithm (e.g., EDF) on each core.
  • EDF mono-processor scheduling algorithm
  • This is a static scheduling approach, i.e., task migration between the cores is not allowed.
  • global scheduling is a dynamic approach. In this case, a single global scheduler determines when and on which core to run the task.
  • task migration is allowed, i.e., each task could run on each processor at any time.
  • a procedure for partitioned scheduling of RT tasks on multicore systems can be used. However, such a procedure is generally tailored for a single node environment, including multiple processors or cores. In contrast, for RT-FaaS, multiple processors of multiple nodes can be managed to execute both RT and non- RT tasks. Consequently, the known RT scheduling algorithms are not applicable to the envisioned RT-FaaS system. [0036] There currently exist certain challenges. While a FaaS system can handle determinism, no known system combines the flexibility of a FaaS system with realtime guarantees. Wrapping RT operations into one FaaS platform is not trivial.
  • the OS of the servers must provide that i) new function deployment requests are processed, ii) the OS related tasks are scheduled, iii) the deployed user functions finish before their deadline requirements and iv) the necessary input and output data of them are transferred in deterministic time between the interface of the user function and the NIC of the server.
  • a RT-FaaS platform may need to provide RT services too (e.g., RT Database) and, to support the microservice approach, it allows functions to call each other. This results in another major challenge that the FaaS must handle, instead of a set of standalone user functions, the input is an interconnected function mesh, the FaaS may need to consider the coherence of them along the multinode multiprocessor scheduling decisions.
  • Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges.
  • Various embodiments herein ensure the realtime execution of user-defined functions in the cloud computing architecture. To achieve this, multiple challenges may need to be resolved, and the solution must take into account the followings: 1) The assumed cloud system is a FaaS in which, except the function code defined by the user, all component is controlled by the platform; 2) The infrastructure on which the FaaS operates is a distributed computing system containing multiple types of servers, NICs, processors, network devices, etc; 3) User demanded functions can call both RT cloud services (like an RT database provided by the FaaS platform) and each other; 4) The function scheduling on the CPUs must consider that processors are distributed among multiple servers.
  • the location of the RT service endpoints and the already deployed function may need to be maintained for future deployment requests; and 5) Also, a big challenge is how to partition the functions among the servers. Because of the distributed nature of the FaaS, functions to deploy must be assigned to servers’ CPU such the RT execution constraints are met.
  • RT-FaaS One of the main challenges that the RT-FaaS must solve is how to schedule the periodically triggered user functions on the multinode multiprocessor system.
  • Prior arts consider a single node environment in which several types of processors exist. In the multinode system, more constraints may need to be kept in mind such that the function migration cannot happen between different nodes, only between other processors of the same node.
  • RT communications between the gateway and the servers must be handled (for the input and output data transmission of the user demanded functions) and also among the servers (if user functions might call each other or cloud provided RT services).
  • some procedures cover only pieces that a real-time FaaS may require. They provide building blocks but not a complete solution.
  • Some embodiments herein aim to attach determinism - besides the dynamism - to the current FaaS platform, thus enabling modern cloud applications to achieve real-time properties, i.e., provide elasticity, resiliency, fast development while keeping runtime deadlines.
  • real-time and non-real-time applications could run together.
  • the deterministic operation of the underlying network and operating systems must be guaranteed, and both spatial and temporal real-time scheduling of user demanded RT functions must be supported.
  • a FaaS system has a deterministic physical (among the servers) and virtual (within a server) network that provides the forwarding of the input data to/from the user demanded function in a given time deadline.
  • RT-FaaS operates servers equipped with real-time OSes, and also the system itself works in real-time, including the RT-FaaS runtime and the offered cloud services to the users like DB, messaging, etc.
  • multinode, heterogeneous multiprocessor scheduling is provided.
  • a distributed task scheduling procedure can be provided in which scheduling happens on heterogeneous processors of multiple servers forming a cluster, guaranteeing to meet completion deadlines for periodically recurrent tasks.
  • an End-to-end (E2E) runtime guarantee for RT functions is provided.
  • the system guarantees that the input data delivery, the demanded function execution, and the output return occur in real-time, i.e., the sum of the necessary time to perform the requested task in FaaS serverless cloud system, is less or equal than the predefined deadline.
  • handling third-party RT cloud service calls or inter-function calls in a deterministic way is provided. If the function calls cloud services, like DB accesses or messaging to another function that runs in the same FaaS system, these requests are handled in real-time. This means the system is aware of the delay that comes with the cloud service call and takes it into account in the function deployment process (to meet its deadline requirement).
  • a complete coordinated FaaS stack offering determinism and dynamism jointly for the user functions is provided.
  • the user may only need to define the RT requirements at deployment time, and the system provides its real-time execution besides the opportunity to scale, upgrade, migrate and perform other cloud features.
  • the proposed cloud design allows for new kinds of use-cases which the current cloud design is unable to implement:
  • Such a usecase could be the remote-controlled factories.
  • Novel applications of Industry 4.0 require reliable wireless connections, ultra-low end-to-end latency, high data rates, and supporting a massive number of devices. If the cloud system guarantees these, there may be no need to run time-critical functions locally, i.e. , it is not necessary to have your own hardware. Opening cloud and edge computing infrastructures for time-critical applications could be an appropriate solution for e.g., cloud-assisted safety systems, tactile internet, virtual and augmented reality.
  • a Latency Quality of Service (“QoS”) guarantees for users is provided.
  • Current cloud systems are not able to guarantee the worst-case execution time of the user demanded function.
  • the proposed RT-FaaS can guarantee the maximal return time which is essential for hard real-time applications.
  • simultaneous execution of RT and non-RT functions can be performed.
  • the proposed RT-FaaS assume a multinode multiprocessor cloud computing system, i.e., a cluster of servers connected by a LAN, and managed by the RT-FaaS platform with the goal of scheduling the user demanded functions such that the RT tasks are assigned to a minimum number of processors of the server cluster which are able to guarantee the deadline requirements are met. This operation leaves as many processors as possible to process the remaining non-RT functions, thus, ensures that they will not starve for the CPU cycle.
  • FIG. 2 presents the building blocks of a proposed RT-FaaS System architecture. Such a system may react to changes anytime during the runtime while ensuring the real-time execution of the requested RT functions.
  • the actor is the user of the RT-FaaS system which or who can be a person, a computer, a function, or a robot hardware.
  • the actor can send his/her/its deployment request to the RT-FaaS platform.
  • the deployment request can include a wrapper entity that includes the function that the actor requests to run. If it is an RT function it can include its realtime requirements.
  • the RT-FaaS Function Deploy Interface can be the entry point of the RT-FaaS system through which the actors can send their deployment requests. This interface can check the syntax of the received requests, and if it is valid, it forwards to the Spatial Function Scheduler component. In the case of syntax error, the interface returns an error message to the actor. [0054] Through the function characteristic change interface, the actor can notify the FaaS in advance about a behaving pattern change of a deployed function. For example, a new robot was installed as a new data source of a function, so the input data rate and the function execution time will change. In this case the Cluster Monitor notifies the Spatial Function Scheduler to take the necessary steps to provide the real-time operation of the given function later on.
  • a Spatial Function Manager can store the incoming deployment request and obtain the requested functions’ worst-case execution time.
  • the spatial function manager can include a function receiving queue to fine-tune the scheduling frequency of the system.
  • the incoming function deployment requests are stored in this queue, and the length of it determines how frequently perform the scheduling process, i.e., how many functions should be scheduled simultaneously.
  • the spatial function manager can assign the RT and non-RT functions to the nodes (and their CPUs) so that RT functions’ deadlines are met and non-RT tasks are not starved.
  • the spatial function manager can assume a multi-node multiprocessor cloud computing system, and after the assignment calculation, the spatial function manager can send the partitioning to the Deployment Controller component. Finally, the component saves the user demanded functions and their attributes to the Function Store.
  • the Deployment Controller can forward the function placements to the affected nodes and configure the nodes to be prepared for the function execution.
  • the latter means it registers the function to the appropriate node’s Node monitor component, forward the function to the Container Configuration Manager and save the function to the Temporal Function Scheduler, and prepares the nodes’ Virtual Network Manager to ensure the communication path of the function inside the nodes. Furthermore, it registers the function to the Physical Network Manager, indicating to create a communication path between the FaaS system gateway and the user demanded function.
  • a Cluster Monitor can collect all the monitored features of the physical nodes and the deployed functions in the underlying FaaS infrastructure and summarize them into a model. If any function or node characteristic change happens, it notifies the Spatial Function Scheduler about the change, thus initializing a new function scheduling to ensure the real-time requirements.
  • a Node monitor can be responsible for monitoring everything that happens inside the node, e.g., maintaining the number of functions running on the hosts, tracking the input and output data volumes and rates, measuring the virtual network performance, and the number of connections between pods and pods-to- services.
  • a Container Configuration Manager can configure and launch the appropriate containers that are suitable for running the RT and/or non-RT functions. Also, this assigns the given functions to the associated processors decided by the Spatial Function Scheduler.
  • a Temporal Function Scheduler can decide which function- among the associated ones to the node- and when to run on one of the node’s processors in a way where the deadline requirements (if exist) are met (e.g., such a real-time scheduler may be the EDF).
  • This scheduler either could be implemented by the OS’s CPU scheduler (similarly to the hypervisor approach) or could be a separated one. In the latter case, there are CPUs that execute only user functions and not system-related processes. The functions may belong to multiple tenants.
  • a Virtual Network Manager configures and sets up the RT virtual network within the node between the launched containers - including the user demanded functions - and the physical network connected to the nodes to each other. For example, this component starts the virtual network devices, connects the containers, ensures the RT connections, configure the NIC’s traffic queues, classify the traffic types and connect the physical NIC port to the virtual network.
  • a Physical Network Manager connects the hosts to the gateway of the FaaS and each other. This manager’s responsibility to ensure enough bandwidth, latency, and the packet loss of data transfer between deployed - user demanded - function and the edge of the physical network, i.e. , the Gateway device. Thanks to this manager, the communication latency is upper bounded, that is the necessary worst-case time for sending data from one point to another can be calculated quite easily.
  • a Network Monitor monitors the underlying physical network attributes, such as measuring transmission times among servers and between server and gateway or maintaining the latency and the currently available bandwidths. If something modification occurs, the module reports it to the Cluster Monitor, which will notify the Spatial Function Scheduler to guarantee the deterministic behaving of the deployed RT functions.
  • a RT FaaS Gateway Monitor is the edge of the data plane of the RT- FaaS system.
  • the deployed tasks can communicate with the internet through this network device.
  • the real-time characteristic of the system is guaranteed only behind this module of the RT-FaaS system (e.g., communication through the internet cannot be controlled).
  • This module detects if a new input data arrives and forwards it to the proper host where the task was initially deployed. Also, it measures the rates of the deployed tasks’ input, the used bandwidths, the number of data sources, the sent data rates and quantities. If a pattern change happens - that could affect the determinism of the deployed function - the module reports it to the Cluster Monitor.
  • FIG. 3 illustrates a possible architecture of a server that operates in the infrastructure of the proposed RT-FaaS cloud. All servers run a real-time OS to be able to execute the user demanded functions. The functions are executed as processes in the operating system, so the OS runs RT and non-RT functions jointly. All function is wrapped into a container including a virtual interface (VNIC) through which the input and output data pass.
  • VNIC virtual interface
  • the RT functions running in the RT- FaaS can be considered to be periodic tasks, meaning that the processes sleep until the corresponding input data is available in the server.
  • All servers contain at least one NIC (Network Interface Controller) through which the traffic for both RT and non-RT functions arrives and departs from/to the physical network.
  • the virtual network is between the physical NIC and the virtual NIC (VNIC) of the containers.
  • the RT Messaging system is aware that if an input data has arrived to an RT function, it sets the state of the particular process to ready for execution.
  • the RT processes are assigned to different processors (one processor can be assigned to multiple processes, but one process can be assigned to only one processor).
  • the Spatial Function Scheduler provides this assignment (see in FIG. 2). All CPUs assigned to RT processes own a CPU RT scheduler (e.g., EDF).
  • This scheduler decides which process (with ready for execution state) when will run to meet their deadline requirements.
  • the containers running non-RT processes use a non-RT Messaging system that enables the traditional best-effort networking.
  • Non-RT functions are executed by any of the remaining processors which are not assigned to RT functions.
  • the multiprocessor scheduler schedules these CPUs.
  • CPUs are distinguished according to execute RT or non-RT functions. A standalone single CPU RT scheduler exists for the formers, while for the latters a multiprocessor non-RT scheduler stands.
  • a Server Resource Monitor and Manager component is responsible for the monitoring (e.g., counting the number of functions running on the hosts), configuring which CPU is scheduled by a local CPU RT Scheduler like EDF and assigning the RT-tasks to the CPUs.
  • a possible scheduling solution can be a time-consuming algorithm that returns optimal placement of the functions.
  • the problem can be formulated as an ILP (Integer Linear Programming) that can be given as an input to an ILP solver program.
  • ILP Intelligent Linear Programming
  • the ILP solvers might solve the given program in non-polynomial time, it always finds the optimal solution.
  • the ILP formalization of finding processor function assignments, such that all function deadline requirements are met, might be the following.
  • the server cluster contains multiple processors (TT1 , TT2, . . . , Tim).
  • the functions to deploy and run in real-time are depicts by (T1 , T2, ... , m).
  • the two variable types are for 1 ⁇ i ⁇ n and 1 ⁇ j ⁇ m.
  • the former denotes whether function Ti is assigned to processor TTJ
  • latter indicates if any of the functions were assigned to processor TTJ.
  • V j 1.. .
  • a heuristic algorithm that distributes the given function set among the processors of cluster hosts is implemented.
  • the utilization matrix s columns represent the processors, while rows correspond to the functions that the user wants to run.
  • On cell of the utilization matrix (e.g., u ⁇ 1 ,2 ⁇ ) represents the CPU utilization of function T1 if it is assigned to processor TT2.
  • exclusive task Such a task that could be run by only one specific processor then perform at least one of: 1 ) pick its processor in M’, regardless its weight, and assign all the one elements (tasks) of the processor’s column to the processor; 2) Assign all tasks which have 1 value in the picked processor’s column to the picked CPU; and 3) Delete the assigned tasks and processor, i.e., remove the proper rows and column from M and start again with Step 1 if still row exists.
  • an unassigned task exists then perform at least one of: 1 ) Pick the processor in M’ with the maximal weight - if multiple exist choose randomly - and assign the one elements (tasks) to it in the partitioning; 2) Assign all tasks which have 1 value in the picked processor’s column to the picked CPU; and 3) Delete the assigned tasks’ rows and the processor’s column from M and start again.
  • ALG is an assignment of functions to processors such that the EDF schedulers of the processors can guarantee the real-time completion of them.
  • FIG. 2 presents the system architecture of the proposed RT-FaaS. Naturally, it works in a distributed system, that is in a cluster of servers and the network among them.
  • FIG. 5 depicts the physical device elements with grey squares and assign the proposed components to the devices in a distributed manner.
  • the network node may be any of the network node 710A, 710B, 900, 1206, hardware 1 104, or virtual machine 1108A, 1 108B
  • the network node 900 shall be used to describe the functionality of the operations of the network node. Operations of the network node 900 (implemented using the structure of FIG. 9) will now be discussed with reference to the flow chart of FIG.
  • modules may be stored in memory 904 of FIG. 9, and these modules may provide instructions so that when the instructions of a module are executed by respective network node processing circuitry 902, processing circuitry 902 performs respective operations of the flow chart.
  • FIG. 6 illustrates an example of operations performed by a network node of multinode cloud-based system.
  • the multinode cloud-based system includes a FaaS system and the network node includes a spatial function scheduler.
  • processing circuitry 902 receives, via communication interface 906, a deployment request for an application function, AF.
  • receiving the deployment request includes receiving the deployment request from a remote device via an AF deployment interface of the multinode cloudbased system.
  • processing circuitry 902 determines requirements of the AF.
  • determining the requirements of the AF include determining whether the AF is a RT AF or a non-real time (“NRT”) AF.
  • determining the requirements of the AF includes determining a latency threshold associated with the AF.
  • the latency threshold associated with a NRT AF can be a best effort threshold in which attempts are made to keep as low of a latency as possible.
  • processing circuitry 902 determines information associated with the multinode cloud-based system. In some embodiments, determining the information associated with the multinode cloud-based system includes receiving an indication of the information from a cluster monitor of the multinode cloud-based system. In additional or alternative embodiments, the information includes at least one of: processing latency, network latency, and service latency associated with each resource of a plurality of resources of the multinode cloud-based system. [0087] At block 640, processing circuitry 902 determines a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloudbased system.
  • determining the resource includes determining the resource based on the resource being capable of keeping a latency associated with handling the AF below a latency threshold.
  • determining the resource to be used to handle the AF includes determining the resource based on the resource being capable of keeping a latency associated with handling another AF below a first threshold and keeping a latency associated with handling the AF below a second threshold. In some examples, being below the first threshold can correspond to being a lowest latency possible.
  • determining the resource to be used to handle the AF includes determining the resource based on the resource being capable of keeping the lowest latency possible associated with handling another AF (e.g., a NRT AF) and keeping a latency associated with handling the AF (e.g., a RT AF) below a second threshold.
  • a NRT AF e.g., a NRT AF
  • a latency associated with handling the AF e.g., a RT AF
  • determining the resource includes determining the resource to ensure an end-to-end runtime of each AF of a plurality of AFs handled by the multinode cloud-based system are below respective threshold values.
  • processing circuitry 902 transmits, via communication interface 906, instructions indicating the resource to be used to handle the AF.
  • instructions indicating the resource to be used to handle the AF.
  • Various operations from the flow chart of FIG. 6 may be optional with respect to some embodiments of network nodes and related methods. For example, operations of block 650 may be optional.
  • FIG. 7 shows an example of a communication system 700 in accordance with some embodiments.
  • the communication system 700 includes a telecommunication network 702 that includes an access network 704, such as a radio access network (RAN), and a core network 706, which includes one or more core network nodes 708.
  • the access network 704 includes one or more access network nodes, such as network nodes 710a and 710b (one or more of which may be generally referred to as network nodes 710), or any other similar 3 rd Generation Partnership Project (3GPP) access node or non-3GPP access point.
  • 3GPP 3 rd Generation Partnership Project
  • the network nodes 710 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 712a, 712b, 712c, and 712d (one or more of which may be generally referred to as UEs 712) to the core network 706 over one or more wireless connections.
  • UE user equipment
  • Example wireless communications over a wireless connection include transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors.
  • the communication system 700 may include any number of wired or wireless networks, network nodes, UEs, and/or any other components or systems that may facilitate or participate in the communication of data and/or signals whether via wired or wireless connections.
  • the communication system 700 may include and/or interface with any type of communication, telecommunication, data, cellular, radio network, and/or other similar type of system.
  • the UEs 712 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and/or operable to communicate wirelessly with the network nodes 710 and other communication devices.
  • the network nodes 710 are arranged, capable, configured, and/or operable to communicate directly or indirectly with the UEs 712 and/or with other network nodes or equipment in the telecommunication network 702 to enable and/or provide network access, such as wireless network access, and/or to perform other functions, such as administration in the telecommunication network 702.
  • the core network 706 connects the network nodes 710 to one or more hosts, such as host 716. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts.
  • the core network 706 includes one more core network nodes (e.g., core network node 708) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and/or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 708.
  • Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and/or a User Plane Function (UPF).
  • MSC Mobile Switching Center
  • MME Mobility Management Entity
  • HSS Home Subscriber Server
  • AMF Access and Mobility Management Function
  • SMF Session Management Function
  • AUSF Authentication Server Function
  • SIDF Subscription Identifier De-concealing function
  • UDM Unified Data Management
  • SEPP Security Edge Protection Proxy
  • NEF Network Exposure Function
  • UPF User Plane Function
  • the host 716 may be under the ownership or control of a service provider other than an operator or provider of the access network 704 and/or the telecommunication network 702, and may be operated by the service provider or on behalf of the service provider.
  • the host 716 may host a variety of applications to provide one or more service. Examples of such applications include live and prerecorded audio/video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.
  • the communication system 700 of FIG. 7 enables connectivity between the UEs, network nodes, and hosts.
  • the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and/or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and/or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.
  • GSM Global System for Mobile Communications
  • UMTS Universal Mobile Telecommunications System
  • LTE Long Term Evolution
  • the telecommunication network 702 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 702 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 702. For example, the telecommunications network 702 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and/or Massive Machine Type Communication (mMTC)/Massive loT services to yet further UEs.
  • URLLC Ultra Reliable Low Latency Communication
  • eMBB Enhanced Mobile Broadband
  • mMTC Massive Machine Type Communication
  • the UEs 712 are configured to transmit and/or receive information without direct human interaction.
  • a UE may be designed to transmit information to the access network 704 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 704.
  • a UE may be configured for operating in single- or multi- RAT or multi-standard mode.
  • a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).
  • MR-DC multi-radio dual connectivity
  • the hub 714 communicates with the access network 704 to facilitate indirect communication between one or more UEs (e.g., UE 712c and/or 712d) and network nodes (e.g., network node 710b).
  • the hub 714 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs.
  • the hub 714 may be a broadband router enabling access to the core network 706 for the UEs.
  • the hub 714 may be a controller that sends commands or instructions to one or more actuators in the UEs.
  • the hub 714 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data.
  • the hub 714 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 714 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 714 then provides to the UE either directly, after performing local processing, and/or after adding additional local content.
  • the hub 714 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.
  • the hub 714 may have a constant/persistent or intermittent connection to the network node 710b.
  • the hub 714 may also allow for a different communication scheme and/or schedule between the hub 714 and UEs (e.g., UE 712c and/or 712d), and between the hub 714 and the core network 706.
  • the hub 714 is connected to the core network 706 and/or one or more UEs via a wired connection.
  • the hub 714 may be configured to connect to an M2M service provider over the access network 704 and/or to another UE over a direct connection.
  • UEs may establish a wireless connection with the network nodes 710 while still connected via the hub 714 via a wired or wireless connection.
  • the hub 714 may be a dedicated hub - that is, a hub whose primary function is to route communications to/from the UEs from/to the network node 710b.
  • the hub 714 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 710b, but which is additionally capable of operating as a communication start and/or end point for certain data channels.
  • FIG. 8 shows a UE 800 in accordance with some embodiments.
  • a UE refers to a device capable, configured, arranged and/or operable to communicate wirelessly with network nodes and/or other UEs.
  • Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded/integrated wireless device, etc.
  • VoIP voice over IP
  • LME laptop-embedded equipment
  • LME laptop-mounted equipment
  • CPE wireless customer-premise equipment
  • UEs identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-loT) UE, a machine type communication (MTC) UE, and/or an enhanced MTC (eMTC) UE.
  • 3GPP 3rd Generation Partnership Project
  • NB-loT narrow band internet of things
  • MTC machine type communication
  • eMTC enhanced MTC
  • a UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X).
  • D2D device-to-device
  • DSRC Dedicated Short-Range Communication
  • V2V vehicle-to-vehicle
  • V2I vehicle-to-infrastructure
  • V2X vehicle-to-everything
  • a UE may not necessarily have a user in the sense of a human user who owns and/or operates the relevant device.
  • a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller).
  • a UE may represent a device that is not intended for sale
  • the UE 800 includes processing circuitry 802 that is operatively coupled via a bus 804 to an input/output interface 806, a power source 808, a memory 810, a communication interface 812, and/or any other component, or any combination thereof.
  • Certain UEs may utilize all or a subset of the components shown in FIG. 8. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.
  • the processing circuitry 802 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 810.
  • the processing circuitry 802 may be implemented as one or more hardware- implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general- purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above.
  • the processing circuitry 802 may include multiple central processing units (CPUs).
  • the input/output interface 806 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and/or output devices.
  • Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof.
  • An input device may allow a user to capture information into the UE 800.
  • Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like.
  • the presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user.
  • a sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof.
  • An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.
  • USB Universal Serial Bus
  • the power source 808 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used.
  • the power source 808 may further include power circuitry for delivering power from the power source 808 itself, and/or an external power source, to the various parts of the UE 800 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 808.
  • Power circuitry may perform any formatting, converting, or other modification to the power from the power source 808 to make the power suitable for the respective components of the UE 800 to which power is supplied.
  • the memory 810 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth.
  • the memory 810 includes one or more application programs 814, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 816.
  • the memory 810 may store, for use by the UE 800, any of a variety of various operating systems or combinations of operating systems.
  • the memory 810 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and/or ISIM, other memory, or any combination thereof.
  • RAID redundant array of independent disks
  • HD-DVD high-density digital versatile disc
  • HDDS holographic digital data storage
  • DIMM external mini-dual in-line memory module
  • SDRAM synchronous dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • the UICC may for example be an embedded UICC (eUlCC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’
  • the memory 810 may allow the UE 800 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data.
  • An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 810, which may be or comprise a device-readable storage medium.
  • the processing circuitry 802 may be configured to communicate with an access network or other network using the communication interface 812.
  • the communication interface 812 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 822.
  • the communication interface 812 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network).
  • Each transceiver may include a transmitter 818 and/or a receiver 820 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth).
  • the transmitter 818 and receiver 820 may be coupled to one or more antennas (e.g., antenna 822) and may share circuit components, software or firmware, or alternatively be implemented separately.
  • communication functions of the communication interface 812 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, nearfield communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof.
  • GPS global positioning system
  • Communications may be implemented in according to one or more communication protocols and/or standards, such as IEEE 802.11 , Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol/internet protocol (TCP/IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.
  • CDMA Code Division Multiplexing Access
  • WCDMA Wideband Code Division Multiple Access
  • GSM Global System for Mobile communications
  • LTE Long Term Evolution
  • NR New Radio
  • UMTS Worldwide Interoperability for Microwave Access
  • WiMax Ethernet
  • TCP/IP transmission control protocol/internet protocol
  • SONET synchronous optical networking
  • ATM Asynchronous Transfer Mode
  • QUIC Hypertext Transfer Protocol
  • HTTP Hypertext Transfer Protocol
  • a UE may provide an output of data captured by its sensors, through its communication interface 812, via a wireless connection to a network node.
  • Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE.
  • the output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).
  • a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection.
  • the states of the actuator, the motor, or the switch may change.
  • the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.
  • a UE when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare.
  • loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door/window sensor, a flood/moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal-
  • AR Augmented Reality
  • VR
  • a UE may represent a machine or other device that performs monitoring and/or measurements, and transmits the results of such monitoring and/or measurements to another UE and/or a network node.
  • the UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device.
  • the UE may implement the 3GPP NB-loT standard.
  • a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and/or reporting on its operational status or other functions associated with its operation.
  • any number of UEs may be used together with respect to a single use case.
  • a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone.
  • the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed.
  • the first and/or the second UE can also include more than one of the functionalities described above.
  • a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.
  • FIG. 9 shows a network node 900 in accordance with some embodiments.
  • network node refers to equipment capable, configured, arranged and/or operable to communicate directly or indirectly with a UE and/or with other network nodes or equipment, in a telecommunication network.
  • network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).
  • APs access points
  • BSs base stations
  • Node Bs Node Bs
  • eNBs evolved Node Bs
  • gNBs NR NodeBs
  • Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations.
  • a base station may be a relay node or a relay donor node controlling a relay.
  • a network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and/or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio.
  • RRUs remote radio units
  • RRHs Remote Radio Heads
  • Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio.
  • Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).
  • DAS distributed antenna system
  • network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell/multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, SelfOrganizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and/or Minimization of Drive Tests (MDTs).
  • MSR multi-standard radio
  • RNCs radio network controllers
  • BSCs base station controllers
  • BTSs base transceiver stations
  • OFDM Operation and Maintenance
  • OSS Operations Support System
  • SON SelfOrganizing Network
  • positioning nodes e.g., Evolved Serving Mobile Location Centers (E-SMLCs)
  • the network node 900 includes a processing circuitry 902, a memory 904, a communication interface 906, and a power source 908.
  • the network node 900 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components.
  • the network node 900 comprises multiple separate components (e.g., BTS and BSC components)
  • one or more of the separate components may be shared among several network nodes.
  • a single RNC may control multiple NodeBs.
  • each unique NodeB and RNC pair may in some instances be considered a single separate network node.
  • the network node 900 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 904 for different RATs) and some components may be reused (e.g., a same antenna 910 may be shared by different RATs).
  • the network node 900 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 900, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 900.
  • RFID Radio Frequency Identification
  • the processing circuitry 902 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and/or encoded logic operable to provide, either alone or in conjunction with other network node 900 components, such as the memory 904, to provide network node 900 functionality.
  • the processing circuitry 902 includes a system on a chip (SOC). In some embodiments, the processing circuitry 902 includes one or more of radio frequency (RF) transceiver circuitry 912 and baseband processing circuitry 914. In some embodiments, the radio frequency (RF) transceiver circuitry 912 and the baseband processing circuitry 914 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 912 and baseband processing circuitry 914 may be on the same chip or set of chips, boards, or units.
  • SOC system on a chip
  • the processing circuitry 902 includes one or more of radio frequency (RF) transceiver circuitry 912 and baseband processing circuitry 914.
  • the radio frequency (RF) transceiver circuitry 912 and the baseband processing circuitry 914 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of
  • the memory 904 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid- state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or nonvolatile, non-transitory device-readable and/or computer-executable memory devices that store information, data, and/or instructions that may be used by the processing circuitry 902.
  • volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid- state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or nonvola
  • the memory 904 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and/or other instructions capable of being executed by the processing circuitry 902 and utilized by the network node 900.
  • the memory 904 may be used to store any calculations made by the processing circuitry 902 and/or any data received via the communication interface 906.
  • the processing circuitry 902 and memory 904 is integrated.
  • the communication interface 906 is used in wired or wireless communication of signaling and/or data between a network node, access network, and/or UE. As illustrated, the communication interface 906 comprises port(s)/terminal(s) 916 to send and receive data, for example to and from a network over a wired connection.
  • the communication interface 906 also includes radio frontend circuitry 918 that may be coupled to, or in certain embodiments a part of, the antenna 910. Radio front-end circuitry 918 comprises filters 920 and amplifiers 922.
  • the radio front-end circuitry 918 may be connected to an antenna 910 and processing circuitry 902.
  • the radio front-end circuitry may be configured to condition signals communicated between antenna 910 and processing circuitry 902.
  • the radio front-end circuitry 918 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection.
  • the radio front-end circuitry 918 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 920 and/or amplifiers 922.
  • the radio signal may then be transmitted via the antenna 910.
  • the antenna 910 may collect radio signals which are then converted into digital data by the radio front-end circuitry 918.
  • the digital data may be passed to the processing circuitry 902.
  • the communication interface may comprise different components and/or different combinations of components.
  • the network node 900 does not include separate radio front-end circuitry 918, instead, the processing circuitry 902 includes radio front-end circuitry and is connected to the antenna 910. Similarly, in some embodiments, all or some of the RF transceiver circuitry 912 is part of the communication interface 906. In still other embodiments, the communication interface 906 includes one or more ports or terminals 916, the radio front-end circuitry 918, and the RF transceiver circuitry 912, as part of a radio unit (not shown), and the communication interface 906 communicates with the baseband processing circuitry 914, which is part of a digital unit (not shown).
  • the antenna 910 may include one or more antennas, or antenna arrays, configured to send and/or receive wireless signals.
  • the antenna 910 may be coupled to the radio front-end circuitry 918 and may be any type of antenna capable of transmitting and receiving data and/or signals wirelessly.
  • the antenna 910 is separate from the network node 900 and connectable to the network node 900 through an interface or port.
  • the antenna 910, communication interface 906, and/or the processing circuitry 902 may be configured to perform any receiving operations and/or certain obtaining operations described herein as being performed by the network node. Any information, data and/or signals may be received from a UE, another network node and/or any other network equipment. Similarly, the antenna 910, the communication interface 906, and/or the processing circuitry 902 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and/or signals may be transmitted to a UE, another network node and/or any other network equipment.
  • the power source 908 provides power to the various components of network node 900 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component).
  • the power source 908 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 900 with power for performing the functionality described herein.
  • the network node 900 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 908.
  • the power source 908 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.
  • Embodiments of the network node 900 may include additional components beyond those shown in FIG. 9 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and/or any functionality necessary to support the subject matter described herein.
  • the network node 900 may include user interface equipment to allow input of information into the network node 900 and to allow output of information from the network node 900. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 900.
  • FIG. 10 is a block diagram of a host 1000, which may be an embodiment of the host 716 of FIG. 7, in accordance with various aspects described herein.
  • the host 1000 may be or comprise various combinations hardware and/or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm.
  • the host 1000 may provide one or more services to one or more UEs.
  • the host 1000 includes processing circuitry 1002 that is operatively coupled via a bus 1004 to an input/output interface 1006, a network interface 1008, a power source 1010, and a memory 1012.
  • Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as FIGS. 8 and 9, such that the descriptions thereof are generally applicable to the corresponding components of host 1000.
  • the memory 1012 may include one or more computer programs including one or more host application programs 1014 and data 1016, which may include user data, e.g., data generated by a UE for the host 1000 or data generated by the host 1000 for a UE.
  • Embodiments of the host 1000 may utilize only a subset or all of the components shown.
  • the host application programs 1014 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems).
  • the host application programs 1014 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network.
  • the host 1000 may select and/or indicate a different host for over-the-top services for a UE.
  • the host application programs 1014 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.
  • HLS HTTP Live Streaming
  • RTMP Real-Time Messaging Protocol
  • RTSP Real-Time Streaming Protocol
  • MPEG-DASH Dynamic Adaptive Streaming over HTTP
  • FIG. 11 is a block diagram illustrating a virtualization environment 1100 in which functions implemented by some embodiments may be virtualized.
  • virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources.
  • virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components.
  • Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1100 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host.
  • VMs virtual machines
  • the virtual node does not require radio connectivity (e.g., a core network node or host)
  • the node may be entirely virtualized.
  • Applications 1102 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and/or benefits of some of the embodiments disclosed herein.
  • Hardware 1104 includes processing circuitry, memory that stores software and/or instructions executable by hardware processing circuitry, and/or other hardware devices as described herein, such as a network interface, input/output interface, and so forth.
  • Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1106 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1108a and 1108b (one or more of which may be generally referred to as VMs 1108), and/or perform any of the functions, features and/or benefits described in relation with some embodiments described herein.
  • the virtualization layer 1106 may present a virtual operating platform that appears like networking hardware to the VMs 1108.
  • the VMs 1108 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1106.
  • a virtualization layer 1106 Different embodiments of the instance of a virtual appliance 1102 may be implemented on one or more of VMs 1108, and the implementations may be made in different ways.
  • Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.
  • NFV network function virtualization
  • a VM 1108 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, nonvirtualized machine.
  • Each of the VMs 1108, and that part of hardware 1104 that executes that VM be it hardware dedicated to that VM and/or hardware shared by that VM with others of the VMs, forms separate virtual network elements.
  • a virtual network function is responsible for handling specific network functions that run in one or more VMs 1108 on top of the hardware 1104 and corresponds to the application 1102.
  • Hardware 1104 may be implemented in a standalone network node with generic or specific components. Hardware 1104 may implement some functions via virtualization. Alternatively, hardware 1104 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1110, which, among others, oversees lifecycle management of applications 1102.
  • hardware 1104 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station.
  • some signaling can be provided with the use of a control system 1112 which may alternatively be used for communication between hardware nodes and radio units.
  • FIG. 12 shows a communication diagram of a host 1202 communicating via a network node 1204 with a UE 1206 over a partially wireless connection in accordance with some embodiments.
  • host 1202 Like host 1000, embodiments of host 1202 include hardware, such as a communication interface, processing circuitry, and memory.
  • the host 1202 also includes software, which is stored in or accessible by the host 1202 and executable by the processing circuitry.
  • the software includes a host application that may be operable to provide a service to a remote user, such as the UE 1206 connecting via an over-the-top (OTT) connection 1250 extending between the UE 1206 and host 1202.
  • OTT over-the-top
  • a host application may provide user data which is transmitted using the OTT connection 1250.
  • the network node 1204 includes hardware enabling it to communicate with the host 1202 and UE 1206.
  • the connection 1260 may be direct or pass through a core network (like core network 706 of FIG. 7) and/or one or more other intermediate networks, such as one or more public, private, or hosted networks.
  • a core network like core network 706 of FIG. 7
  • one or more other intermediate networks such as one or more public, private, or hosted networks.
  • an intermediate network may be a backbone network or the Internet.
  • the UE 1206 includes hardware and software, which is stored in or accessible by UE 1206 and executable by the UE’s processing circuitry.
  • the software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1206 with the support of the host 1202.
  • a client application such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1206 with the support of the host 1202.
  • an executing host application may communicate with the executing client application via the OTT connection 1250 terminating at the UE 1206 and host 1202.
  • the UE's client application may receive request data from the host's host application and provide user data in response to the request data.
  • the OTT connection 1250 may transfer both the request data and the user data.
  • the UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT
  • the OTT connection 1250 may extend via a connection 1260 between the host 1202 and the network node 1204 and via a wireless connection 1270 between the network node 1204 and the UE 1206 to provide the connection between the host 1202 and the UE 1206.
  • the connection 1260 and wireless connection 1270, over which the OTT connection 1250 may be provided, have been drawn abstractly to illustrate the communication between the host 1202 and the UE 1206 via the network node 1204, without explicit reference to any intermediary devices and the precise routing of messages via these devices.
  • the host 1202 provides user data, which may be performed by executing a host application.
  • the user data is associated with a particular human user interacting with the UE 1206.
  • the user data is associated with a UE 1206 that shares data with the host 1202 without explicit human interaction.
  • the host 1202 initiates a transmission carrying the user data towards the UE 1206.
  • the host 1202 may initiate the transmission responsive to a request transmitted by the UE 1206.
  • the request may be caused by human interaction with the UE 1206 or by operation of the client application executing on the UE 1206.
  • the transmission may pass via the network node 1204, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1212, the network node 1204 transmits to the UE 1206 the user data that was carried in the transmission that the host 1202 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1214, the UE 1206 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1206 associated with the host application executed by the host 1202.
  • the UE 1206 executes a client application which provides user data to the host 1202.
  • the user data may be provided in reaction or response to the data received from the host 1202.
  • the UE 1206 may provide user data, which may be performed by executing the client application.
  • the client application may further consider user input received from the user via an input/output interface of the UE 1206. Regardless of the specific manner in which the user data was provided, the UE 1206 initiates, in step 1218, transmission of the user data towards the host 1202 via the network node 1204.
  • the network node 1204 receives user data from the UE 1206 and initiates transmission of the received user data towards the host 1202.
  • the host 1202 receives the user data carried in the transmission initiated by the UE 1206.
  • One or more of the various embodiments improve the performance of OTT services provided to the UE 1206 using the OTT connection 1250, in which the wireless connection 1270 forms the last segment. More precisely, the teachings of these embodiments may allow a multinode cloud-based system (e.g., a FaaS system) to schedule functions based on requirements of the function and a status of the cloud-based system, and thereby ensure E2E RT runtimes for RT functions.
  • factory status information may be collected and analyzed by the host 1202.
  • the host 1202 may process audio and video data which may have been retrieved from a UE for use in creating maps.
  • the host 1202 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights).
  • the host 1202 may store surveillance video uploaded by a UE.
  • the host 1202 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs.
  • the host 1202 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and/or transmitting data.
  • a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve.
  • the measurement procedure and/or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1202 and/or UE 1206.
  • sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1250 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities.
  • the reconfiguring of the OTT connection 1250 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1204. Such procedures and functionalities may be known and practiced in the art.
  • measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1202.
  • the measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 1250 while monitoring propagation times, errors, etc.
  • computing devices described herein may include the illustrated combination of hardware components
  • computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components.
  • a communication interface may be configured to include any of the components described herein, and/or the functionality of the components may be partitioned between the processing circuitry and the communication interface.
  • non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.
  • processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium.
  • some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner.
  • the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and/or by end users and a wireless network generally.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Multi Processors (AREA)

Abstract

A network node of a multinode cloud-based system can receive (910) a deployment request for an application function ("AF"). The network node can determine (920) requirements of the AF including a latency threshold associated with the AF. The network node can determine (930) information associated with the multinode cloud-based system. The network node can determine (940) a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloud-based system.

Description

REAL-TIME FUNCTION-AS-A-SERVICE SYSTEM
TECHNICAL FIELD
[0001 ] The present disclosure is related to wireless communication systems and more particularly to a real-time function-as-a-service system.
BACKGROUND
[0002] FIG. 1 illustrates an example of a new radio (“NR”) network (e.g., a 5th Generation (“5G”) network) including a 5G core (“5GC”) network 130, network nodes 120a-b (e.g., 5G base station (“gNB”)), multiple communication devices 1 10 (also referred to as user equipment (“UE”)).
[0003] Function-as-a-Service (“FaaS”) is a concept of serverless cloud computing. FaaS can make it possible for developers to focus on application development without any system management effort. FaaS can ensure resource allocation, fast response time, schedulability, scalability, resiliency and upgradability. Applications of 5G, internet of things (‘ loT”, and Industry 4.0 raise the idea to open cloud-edge computing infrastructures for time critical applications too. Therefore, there is a desire to pose real-time (“RT”) requirements for computing systems like FaaS.
SUMMARY
[0004] According to some embodiments, a method performed by a network node of a multinode cloud-based system is provided. The method includes receiving a deployment request for an application function (“AF”). The method further includes determining requirements of the AF including a latency threshold associated with the AF. The method further includes determining information associated with the multinode cloud-based system. The method further includes determining a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloudbased system.
[0005] According to other embodiments, a network node, computer program, computer program product, or non-transitory computer-readable medium is provided to perform the above method. [0006] Certain embodiments may provide one or more of the following technical advantages. In some embodiments, the proposed cloud design allows for new kinds of use-cases which the current cloud design is unable to implement: Such a usecase could be the remote-controlled factories. Novel applications of Industry 4.0 require reliable wireless connections, ultra-low end-to-end latency, high data rates, and supporting a massive number of devices. If the cloud system guarantees these, there is may be no need to run time-critical functions locally, i.e., it is not necessary to have your own hardware. Opening cloud and edge computing infrastructures for time-critical applications could be an appropriate solution for e.g., cloud-assisted safety systems, tactile internet, virtual and augmented reality.
[0007] In additional or alternative embodiments, a Latency Quality of Service (“QoS”) guarantees for users is provided. Current cloud systems are not able to guarantee the worst-case execution time of the user demanded function. The proposed RT-FaaS can guarantee the maximal return time which is essential for hard real-time applications.
[0008] In additional or alternative embodiments, simultaneous execution of RT and non-RT functions can be performed. The proposed RT-FaaS assume a multinode multiprocessor cloud computing system, i.e., a cluster of servers connected by a LAN, and managed by the RT-FaaS platform with the goal of scheduling the user demanded functions such that the RT tasks are assigned to a minimum number of processors of the server cluster which are able to guarantee the deadline requirements are met. This operation leaves as many processors as possible to process the remaining non-RT functions, thus, ensures that they will not starve for the CPU cycle.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of inventive concepts. In the drawings:
[0010] FIG. 1 is a schematic diagram illustrating an example of a 5th generation (“5G”) network;
[0011 ] FIG. 2 is a block diagram illustrating an example of a system architecture according to some embodiments; [0012] FIG. 3 is a block diagram illustrating an example of components of a server managed by RT-FaaS according to some embodiments;
[0013] FIG. 4 is a flow chart illustrating an example of a partitioning procedure according to some embodiments;
[0014] FIG. 5 is a block diagram illustrating an example of a distributed system architecture according to some embodiments;
[0015] FIG. 6 is a flow chart illustrating an example of operations of a network node according to some embodiments of inventive concepts;
[0016] FIG. 7 is a block diagram of a communication system in accordance with some embodiments;
[0017] FIG. 8 is a block diagram of a user equipment in accordance with some embodiments
[0018] FIG. 9 is a block diagram of a network node in accordance with some embodiments;
[0019] FIG. 10 is a block diagram of a host computer communicating with a user equipment in accordance with some embodiments;
[0020] FIG. 11 is a block diagram of a virtualization environment in accordance with some embodiments; and
[0021 ] FIG. 12 is a block diagram of a host computer communicating via a base station with a user equipment over a partially wireless connection in accordance with some embodiments in accordance with some embodiments.
DETAILED DESCRIPTION
[0022] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art, in which examples of embodiments of inventive concepts are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of present inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive.
Components from one embodiment may be tacitly assumed to be present/used in another embodiment. [0023] Bringing real-time (“RT”) feature to cloud-based networks can be challenging. The real-time infrastructure must ensure deterministic communication and execution of the applications. Examples of terminology and rules of RT applications include that “the high percentile delay between two application components must be less than x (bounded latencies)” or that “the worst-case execution time (“WCET”) of the function must be y”. What is more, RT systems can require static configuration and operation from both infrastructure and the deployed applications. In some examples, the following must be predetermined: i) the specific hardware lying in the architecture and communicating to each other, ii) the number of them, iii) the network among them, iv) supported protocols, v) the operating systems (“OS”) of the servers, vi) the memory size reserving to the network packets, and vii) the application characteristic. The characteristic defines whether it is an aperiodic, periodic, or sporadic task, its input, and output data rate, its execution depending on the input data or not, how much data source it has.
[0024] By contrast, the essential characteristic of any cloud system (including FaaS) is dynamism. It manages a heterogeneous system (e.g., multiple types of hardware can be included) and manages a continuously changing number of nodes. Geographically distributed servers can form an edge-cloud infrastructure. Furthermore, the user deployed applications have dynamic characteristics such as i) the dynamic number of deployed applications (during runtime applications can be deployed, deleted, or updated anytime), ii) the number of external data sources are changeable, iii) dynamically varying input and output rate and quantity, iv) application execution may depend on the input data, v) various application lifetime in function as a service (“FaaS”) and vi) whether functions are communicating to other applications or services provided by the FaaS.
[0025] The two natures (determinism and dynamism) are in stark contrast. Challenges can arise if the FaaS system and its dynamism are given, for example, it can be challenging to achieve end-to-end determinism in such an environment. Some innovations herein describe a FaaS approach as a cloud system where RT applications could run. In FaaS, each component of the system - such as resource allocation, networking, scheduling, etc. - is controlled by the platform. The users’ only task is to provide the source code of the function to run. All responsibility lies with the FaaS service provider. There is no user-dependent element in the real-time function execution. [0026] Accordingly, some embodiments herein propose a real-time FaaS (“RT- FaaS”) system that combines the determinism of classical real-time systems with the flexibility and development speed of modern cloud platforms.
[0027] Faas platforms are described below. Thanks to its flexibility benefits, serverless computing can be effective in the field of cloud computing. It can allow developers to focus only on application development and not care about the server management and auxiliary functions by providing resource allocation, fast response time, schedulability, scalability, resiliency, and the deployed applications' upgradeability. Function-as-a-Service makes it possible for users to run their microservice applications without any management effort. Most of the FaaS platforms rely on container technologies, like LXC or Docker, and container orchestration systems such as Kubernetes. Typically, the users' source code is wrapped into containers that are scheduled and deployed by Kubernetes. Containerization provides isolation of processes and libraries with low-performance overhead.
[0028] Real-time physical networks are described below. The underlying network of an RT-FaaS system can be highly reliable and provide bounded latency and low jitter. In this field, the two networking standards are IEEE 802.1 TSN and IETF DetNet, which provide deterministic and reliable packet forwarding for link-layer and network-layer services.
[0029] Real-time virtual (within host) networks are described below. In some examples, a Linux communication stack can be meant for real-time robotic applications. Under appropriate configuration, the Linux kernel - which is not optimized for bounded latencies - greatly enhances the deterministic nature of communications if there is no significant non-critical concurrent traffic in the system. If there is, it can disrupt the bounded latencies.
[0030] On the other hand, there exist network stacks with the specific purpose of providing deterministic networking. For example, an open-source hard real-time network protocol stack for Linux equipped with real-time extension RTAI or Xenomai can be provided. UDP/IP, TCP/IP, ICMP, and ARP can be implemented in a deterministic way and RT and non-RT traffic can be handled jointly.
[0031 ] Data flow graphs are popular program representations within a single node environment where graph nodes correspond to processing primitives and graph edges describe control flow. To improve CPU cache locality and exploit data- level parallelism, nodes usually process data in batches. A scheduler for run-to- completion packet processing engines can be provided and can use controlled queuing to efficiently reconstruct fragmented batches in accordance with strict service-level objectives (“SLOs”).
[0032] Xen hypervisor handles real-time and non-real-time traffic through its common interface, but includes limitations that could result in long or unpredictable network traffic latencies between virtual machines (“VMs”). A Virtualization-Aware Traffic Control framework can improve predictability and reduce the delay for latency-sensitive applications.
[0033] Real-time containers are described below. In some examples, critical containers can cohabit with traditional non-real-time containers on the same hardware - on top of a Linux kernel patched with a hard real-time co-kernel. Preemptive fixed-priority scheduling can be selected as the underlying task scheduling. A real-time deadline-based scheduling policy can be built into the Linux kernel to provide temporal scheduling guarantees to different co-located containers. A hierarchical scheduling system that is compatible with the most used containerbased virtualization solutions and extended the SCHED DEADLINE scheduling policy can be provided.
[0034] Real-time scheduling on heterogeneous processors is described below. The multiprocessor scheduling approaches can be divided into two categories: partitioned and global techniques. The former uses a partitioning method to spread the tasks onto processors and runs a mono-processor scheduling algorithm (e.g., EDF) on each core. This is a static scheduling approach, i.e., task migration between the cores is not allowed. On the other hand, global scheduling is a dynamic approach. In this case, a single global scheduler determines when and on which core to run the task. Here task migration is allowed, i.e., each task could run on each processor at any time.
[0035] A procedure for partitioned scheduling of RT tasks on multicore systems can be used. However, such a procedure is generally tailored for a single node environment, including multiple processors or cores. In contrast, for RT-FaaS, multiple processors of multiple nodes can be managed to execute both RT and non- RT tasks. Consequently, the known RT scheduling algorithms are not applicable to the envisioned RT-FaaS system. [0036] There currently exist certain challenges. While a FaaS system can handle determinism, no known system combines the flexibility of a FaaS system with realtime guarantees. Wrapping RT operations into one FaaS platform is not trivial. There are multiple layers (multiprocessor scheduling, physical and virtual networks, and latency-bounded cloud services) of the FaaS system. All of those may need to be deterministic and strongly integrated with each other to ensure a deterministic function execution. Due to the end-to-end (“E2E”) execution time constraint (defined by the user) of the functions, the separated components may need to be synchronized and collaborate with each other as in highly interconnected systems. However, on the other hand, the FaaS dynamism may still need to be ensured. If the FaaS infrastructure is transformed into a highly interconnected system, although the determinism might be ensured, the system's dynamism may be lost. Current approaches bring determinism at the expense of dynamism. The challenge here is ensuring the end-to-end real-time execution while the FaaS dynamism is retained. [0037] Multiple points exist in a FaaS architecture where non-determinism comes into the picture. Since the FaaS system is scattered across multiple servers, one of the fundamental tasks is to ensure RT communication between devices, even if it is a computing server, network bridge, switch, or router. Unfortunately, RT physical network is not enough for the RT function communication since within the servers, the virtual networks also must be RT enabled. Thanks to the shared nature of the CPUs within the server, it is a significant challenge to make that happen. Virtual network devices, OS processes, and the user demanded functions, all compete for CPU resources to be executed. It is still an actively researched field of sharing single or multiple processors so that each component receives the desired CPU cycles and the RT operation of the server is ensured.
[0038] The OS of the servers must provide that i) new function deployment requests are processed, ii) the OS related tasks are scheduled, iii) the deployed user functions finish before their deadline requirements and iv) the necessary input and output data of them are transferred in deterministic time between the interface of the user function and the NIC of the server. Furthermore, a RT-FaaS platform may need to provide RT services too (e.g., RT Database) and, to support the microservice approach, it allows functions to call each other. This results in another major challenge that the FaaS must handle, instead of a set of standalone user functions, the input is an interconnected function mesh, the FaaS may need to consider the coherence of them along the multinode multiprocessor scheduling decisions.
[0039] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges. Various embodiments herein ensure the realtime execution of user-defined functions in the cloud computing architecture. To achieve this, multiple challenges may need to be resolved, and the solution must take into account the followings: 1) The assumed cloud system is a FaaS in which, except the function code defined by the user, all component is controlled by the platform; 2) The infrastructure on which the FaaS operates is a distributed computing system containing multiple types of servers, NICs, processors, network devices, etc; 3) User demanded functions can call both RT cloud services (like an RT database provided by the FaaS platform) and each other; 4) The function scheduling on the CPUs must consider that processors are distributed among multiple servers. Thus, the location of the RT service endpoints and the already deployed function may need to be maintained for future deployment requests; and 5) Also, a big challenge is how to partition the functions among the servers. Because of the distributed nature of the FaaS, functions to deploy must be assigned to servers’ CPU such the RT execution constraints are met.
[0040] One of the main challenges that the RT-FaaS must solve is how to schedule the periodically triggered user functions on the multinode multiprocessor system. Prior arts consider a single node environment in which several types of processors exist. In the multinode system, more constraints may need to be kept in mind such that the function migration cannot happen between different nodes, only between other processors of the same node. In a multinode environment, RT communications between the gateway and the servers must be handled (for the input and output data transmission of the user demanded functions) and also among the servers (if user functions might call each other or cloud provided RT services). [0041 ] As mentioned above, some procedures cover only pieces that a real-time FaaS may require. They provide building blocks but not a complete solution. Some embodiments herein aim to attach determinism - besides the dynamism - to the current FaaS platform, thus enabling modern cloud applications to achieve real-time properties, i.e., provide elasticity, resiliency, fast development while keeping runtime deadlines. In such a system, both real-time and non-real-time applications could run together. Still, for the former, the deterministic operation of the underlying network and operating systems must be guaranteed, and both spatial and temporal real-time scheduling of user demanded RT functions must be supported.
[0042] In some embodiments, deterministic input and output data forwarding of the RT functions are provided. A FaaS system has a deterministic physical (among the servers) and virtual (within a server) network that provides the forwarding of the input data to/from the user demanded function in a given time deadline. RT-FaaS operates servers equipped with real-time OSes, and also the system itself works in real-time, including the RT-FaaS runtime and the offered cloud services to the users like DB, messaging, etc.
[0043] In additional or alternative embodiments, multinode, heterogeneous multiprocessor scheduling is provided. A distributed task scheduling procedure can be provided in which scheduling happens on heterogeneous processors of multiple servers forming a cluster, guaranteeing to meet completion deadlines for periodically recurrent tasks.
[0044] In additional or alternative embodiments, an End-to-end (E2E) runtime guarantee for RT functions is provided. The system guarantees that the input data delivery, the demanded function execution, and the output return occur in real-time, i.e., the sum of the necessary time to perform the requested task in FaaS serverless cloud system, is less or equal than the predefined deadline.
[0045] In additional or alternative embodiments, handling third-party RT cloud service calls or inter-function calls in a deterministic way is provided. If the function calls cloud services, like DB accesses or messaging to another function that runs in the same FaaS system, these requests are handled in real-time. This means the system is aware of the delay that comes with the cloud service call and takes it into account in the function deployment process (to meet its deadline requirement).
[0046] In additional or alternative embodiments, a complete coordinated FaaS stack offering determinism and dynamism jointly for the user functions is provided. The user may only need to define the RT requirements at deployment time, and the system provides its real-time execution besides the opportunity to scale, upgrade, migrate and perform other cloud features.
[0047] Certain embodiments may provide one or more of the following technical advantages. In some embodiments, the proposed cloud design allows for new kinds of use-cases which the current cloud design is unable to implement: Such a usecase could be the remote-controlled factories. Novel applications of Industry 4.0 require reliable wireless connections, ultra-low end-to-end latency, high data rates, and supporting a massive number of devices. If the cloud system guarantees these, there may be no need to run time-critical functions locally, i.e. , it is not necessary to have your own hardware. Opening cloud and edge computing infrastructures for time-critical applications could be an appropriate solution for e.g., cloud-assisted safety systems, tactile internet, virtual and augmented reality.
[0048] In additional or alternative embodiments, a Latency Quality of Service (“QoS”) guarantees for users is provided. Current cloud systems are not able to guarantee the worst-case execution time of the user demanded function. The proposed RT-FaaS can guarantee the maximal return time which is essential for hard real-time applications.
[0049] In additional or alternative embodiments, simultaneous execution of RT and non-RT functions can be performed. The proposed RT-FaaS assume a multinode multiprocessor cloud computing system, i.e., a cluster of servers connected by a LAN, and managed by the RT-FaaS platform with the goal of scheduling the user demanded functions such that the RT tasks are assigned to a minimum number of processors of the server cluster which are able to guarantee the deadline requirements are met. This operation leaves as many processors as possible to process the remaining non-RT functions, thus, ensures that they will not starve for the CPU cycle.
[0050] FIG. 2 presents the building blocks of a proposed RT-FaaS System architecture. Such a system may react to changes anytime during the runtime while ensuring the real-time execution of the requested RT functions.
[0051] The actor is the user of the RT-FaaS system which or who can be a person, a computer, a function, or a robot hardware. The actor can send his/her/its deployment request to the RT-FaaS platform.
[0052] The deployment request can include a wrapper entity that includes the function that the actor requests to run. If it is an RT function it can include its realtime requirements.
[0053] The RT-FaaS Function Deploy Interface can be the entry point of the RT-FaaS system through which the actors can send their deployment requests. This interface can check the syntax of the received requests, and if it is valid, it forwards to the Spatial Function Scheduler component. In the case of syntax error, the interface returns an error message to the actor. [0054] Through the function characteristic change interface, the actor can notify the FaaS in advance about a behaving pattern change of a deployed function. For example, a new robot was installed as a new data source of a function, so the input data rate and the function execution time will change. In this case the Cluster Monitor notifies the Spatial Function Scheduler to take the necessary steps to provide the real-time operation of the given function later on.
[0055] A Spatial Function Manager can store the incoming deployment request and obtain the requested functions’ worst-case execution time. The spatial function manager can include a function receiving queue to fine-tune the scheduling frequency of the system. The incoming function deployment requests are stored in this queue, and the length of it determines how frequently perform the scheduling process, i.e., how many functions should be scheduled simultaneously.
If this queue is full, then it begins to calculate the optimal placement of the functions such that real-time requirements are met.
[0056] The spatial function manager can assign the RT and non-RT functions to the nodes (and their CPUs) so that RT functions’ deadlines are met and non-RT tasks are not starved. The spatial function manager can assume a multi-node multiprocessor cloud computing system, and after the assignment calculation, the spatial function manager can send the partitioning to the Deployment Controller component. Finally, the component saves the user demanded functions and their attributes to the Function Store.
[0057] The Deployment Controller can forward the function placements to the affected nodes and configure the nodes to be prepared for the function execution. The latter means it registers the function to the appropriate node’s Node monitor component, forward the function to the Container Configuration Manager and save the function to the Temporal Function Scheduler, and prepares the nodes’ Virtual Network Manager to ensure the communication path of the function inside the nodes. Furthermore, it registers the function to the Physical Network Manager, indicating to create a communication path between the FaaS system gateway and the user demanded function.
[0058] A Cluster Monitor can collect all the monitored features of the physical nodes and the deployed functions in the underlying FaaS infrastructure and summarize them into a model. If any function or node characteristic change happens, it notifies the Spatial Function Scheduler about the change, thus initializing a new function scheduling to ensure the real-time requirements. [0059] A Node monitor can be responsible for monitoring everything that happens inside the node, e.g., maintaining the number of functions running on the hosts, tracking the input and output data volumes and rates, measuring the virtual network performance, and the number of connections between pods and pods-to- services.
[0060] A Container Configuration Manager can configure and launch the appropriate containers that are suitable for running the RT and/or non-RT functions. Also, this assigns the given functions to the associated processors decided by the Spatial Function Scheduler.
[0061] A Temporal Function Scheduler can decide which function- among the associated ones to the node- and when to run on one of the node’s processors in a way where the deadline requirements (if exist) are met (e.g., such a real-time scheduler may be the EDF). This scheduler either could be implemented by the OS’s CPU scheduler (similarly to the hypervisor approach) or could be a separated one. In the latter case, there are CPUs that execute only user functions and not system-related processes. The functions may belong to multiple tenants.
[0062] A Virtual Network Manager configures and sets up the RT virtual network within the node between the launched containers - including the user demanded functions - and the physical network connected to the nodes to each other. For example, this component starts the virtual network devices, connects the containers, ensures the RT connections, configure the NIC’s traffic queues, classify the traffic types and connect the physical NIC port to the virtual network.
[0063] A Physical Network Manager connects the hosts to the gateway of the FaaS and each other. This manager’s responsibility to ensure enough bandwidth, latency, and the packet loss of data transfer between deployed - user demanded - function and the edge of the physical network, i.e. , the Gateway device. Thanks to this manager, the communication latency is upper bounded, that is the necessary worst-case time for sending data from one point to another can be calculated quite easily.
[0064] A Network Monitor monitors the underlying physical network attributes, such as measuring transmission times among servers and between server and gateway or maintaining the latency and the currently available bandwidths. If something modification occurs, the module reports it to the Cluster Monitor, which will notify the Spatial Function Scheduler to guarantee the deterministic behaving of the deployed RT functions.
[0065] A RT FaaS Gateway Monitor is the edge of the data plane of the RT- FaaS system. The deployed tasks can communicate with the internet through this network device. The real-time characteristic of the system is guaranteed only behind this module of the RT-FaaS system (e.g., communication through the internet cannot be controlled). This module detects if a new input data arrives and forwards it to the proper host where the task was initially deployed. Also, it measures the rates of the deployed tasks’ input, the used bandwidths, the number of data sources, the sent data rates and quantities. If a pattern change happens - that could affect the determinism of the deployed function - the module reports it to the Cluster Monitor.
[0066] An example of a RT FaaS compute server approach is described below. FIG. 3 illustrates a possible architecture of a server that operates in the infrastructure of the proposed RT-FaaS cloud. All servers run a real-time OS to be able to execute the user demanded functions. The functions are executed as processes in the operating system, so the OS runs RT and non-RT functions jointly. All function is wrapped into a container including a virtual interface (VNIC) through which the input and output data pass. The RT functions running in the RT- FaaS can be considered to be periodic tasks, meaning that the processes sleep until the corresponding input data is available in the server.
[0067] All servers contain at least one NIC (Network Interface Controller) through which the traffic for both RT and non-RT functions arrives and departs from/to the physical network. The virtual network is between the physical NIC and the virtual NIC (VNIC) of the containers. The RT Messaging system is aware that if an input data has arrived to an RT function, it sets the state of the particular process to ready for execution. The RT processes are assigned to different processors (one processor can be assigned to multiple processes, but one process can be assigned to only one processor). The Spatial Function Scheduler provides this assignment (see in FIG. 2). All CPUs assigned to RT processes own a CPU RT scheduler (e.g., EDF). This scheduler decides which process (with ready for execution state) when will run to meet their deadline requirements. In contrast to the RT functions, the containers running non-RT processes use a non-RT Messaging system that enables the traditional best-effort networking. Non-RT functions are executed by any of the remaining processors which are not assigned to RT functions. The multiprocessor scheduler schedules these CPUs. In summary, CPUs are distinguished according to execute RT or non-RT functions. A standalone single CPU RT scheduler exists for the formers, while for the latters a multiprocessor non-RT scheduler stands.
[0068] A Server Resource Monitor and Manager component is responsible for the monitoring (e.g., counting the number of functions running on the hosts), configuring which CPU is scheduled by a local CPU RT Scheduler like EDF and assigning the RT-tasks to the CPUs.
[0069] An example of an optimal Spatial Function Scheduler algorithm is described below. A possible scheduling solution can be a time-consuming algorithm that returns optimal placement of the functions. For instance, the problem can be formulated as an ILP (Integer Linear Programming) that can be given as an input to an ILP solver program. Although the ILP solvers might solve the given program in non-polynomial time, it always finds the optimal solution. The ILP formalization of finding processor function assignments, such that all function deadline requirements are met, might be the following. Let us assume that the server cluster contains multiple processors (TT1 , TT2, . . . , Tim). The functions to deploy and run in real-time are depicts by (T1 , T2, ... , m). The two variable types are for 1 < i < n and 1 < j < m. The former denotes whether function Ti is assigned to processor TTJ , while latter indicates if any of the functions were assigned to processor TTJ.
Figure imgf000017_0003
V j — 1.. .
The variables are
Figure imgf000017_0001
1,
T | 1. if r, is assigned to TT,- x^ ~ J (5)
1 0, otherwise if 7T. is used (6)
Figure imgf000017_0002
otherwise
[0070] The explanation for the constraints is the following. Each task must be assigned to exactly one host (2). The cumulative utilization of tasks assigned to the same processor must be less or equal than one due to EDF’s schedulability requirement (3).
[0071 ] It is well known that partitioned scheduling on heterogeneous multiprocessors is intractable in polynomial time, and it is an NP-hard problem, even if processors are identical.
[0072] Another example of a Fast Spatial Function Scheduler algorithm is described below. A fast-running heuristic or approximation algorithms could also find an accurate scheduling decision, but generally, their solution is not optimal. In some embodiments, a heuristic algorithm that distributes the given function set among the processors of cluster hosts is implemented. The input of the algorithm is he I = (n, m, M) triplet i.e. , the number of functions n and processors m, and the utilization matrix M. The utilization matrix’s columns represent the processors, while rows correspond to the functions that the user wants to run. On cell of the utilization matrix (e.g., u{ 1 ,2}) represents the CPU utilization of function T1 if it is assigned to processor TT2.
[0073] The operations of the proposed partitioning algorithm are in FIG. 4 and described below.
[0074] First, convert M to a boolean matrix M’ according to the transformation rule defined in Definition 6 (Converting M to boolean matrix M’: 1) Sort the elements with their original indices in increasing order in each column . 2) Iterate through
Figure imgf000018_0001
indices.).
[0075] Then iterate through the columns of M’ and calculate their weights w(rri), i.e., the number of ones in the column.
[0076] Then consider the best processors in M’ (selecting the best processors: Given two processors TT1 and TT2 of the examined boolean matrix M’. It is said that TT2 is a better processor than TT1 if and only if all 1 items in TT1 are also 1 in TT2 and W(TT2) > W(TT1). AS TT2 is a better processor than TT1, the latter is removed from the columns of M’ . Make this comparison for all column pairs.).
[0077] Then sort the columns of M’ according to their weights in decreasing order.
[0078] If exclusive task exists (Exclusive task: Such a task that could be run by only one specific processor) then perform at least one of: 1 ) pick its processor in M’, regardless its weight, and assign all the one elements (tasks) of the processor’s column to the processor; 2) Assign all tasks which have 1 value in the picked processor’s column to the picked CPU; and 3) Delete the assigned tasks and processor, i.e., remove the proper rows and column from M and start again with Step 1 if still row exists.
[0079] If an unassigned task exists then perform at least one of: 1 ) Pick the processor in M’ with the maximal weight - if multiple exist choose randomly - and assign the one elements (tasks) to it in the partitioning; 2) Assign all tasks which have 1 value in the picked processor’s column to the picked CPU; and 3) Delete the assigned tasks’ rows and the processor’s column from M and start again.
[0080] Then return the task-processor assignments The return value of ALG is an assignment of functions to processors such that the EDF schedulers of the processors can guarantee the real-time completion of them.
[0081 ] While FIG. 2 presents the system architecture of the proposed RT-FaaS. Naturally, it works in a distributed system, that is in a cluster of servers and the network among them. FIG. 5 depicts the physical device elements with grey squares and assign the proposed components to the devices in a distributed manner. [0082] In the description that follows, while the network node may be any of the network node 710A, 710B, 900, 1206, hardware 1 104, or virtual machine 1108A, 1 108B, the network node 900 shall be used to describe the functionality of the operations of the network node. Operations of the network node 900 (implemented using the structure of FIG. 9) will now be discussed with reference to the flow chart of FIG. 6 according to some embodiments of inventive concepts. For example, modules may be stored in memory 904 of FIG. 9, and these modules may provide instructions so that when the instructions of a module are executed by respective network node processing circuitry 902, processing circuitry 902 performs respective operations of the flow chart.
[0083] FIG. 6 illustrates an example of operations performed by a network node of multinode cloud-based system. In some embodiments, the multinode cloud-based system includes a FaaS system and the network node includes a spatial function scheduler.
[0084] At block 610, processing circuitry 902 receives, via communication interface 906, a deployment request for an application function, AF. In some embodiments, receiving the deployment request includes receiving the deployment request from a remote device via an AF deployment interface of the multinode cloudbased system.
[0085] At block 620, processing circuitry 902 determines requirements of the AF. In some embodiments, determining the requirements of the AF include determining whether the AF is a RT AF or a non-real time (“NRT”) AF. In additional or alternative embodiments, determining the requirements of the AF includes determining a latency threshold associated with the AF. In some examples, the latency threshold associated with a NRT AF can be a best effort threshold in which attempts are made to keep as low of a latency as possible.
[0086] At block 630, processing circuitry 902 determines information associated with the multinode cloud-based system. In some embodiments, determining the information associated with the multinode cloud-based system includes receiving an indication of the information from a cluster monitor of the multinode cloud-based system. In additional or alternative embodiments, the information includes at least one of: processing latency, network latency, and service latency associated with each resource of a plurality of resources of the multinode cloud-based system. [0087] At block 640, processing circuitry 902 determines a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloudbased system. In some embodiments, determining the resource includes determining the resource based on the resource being capable of keeping a latency associated with handling the AF below a latency threshold. In additional or alternative embodiments, determining the resource to be used to handle the AF includes determining the resource based on the resource being capable of keeping a latency associated with handling another AF below a first threshold and keeping a latency associated with handling the AF below a second threshold. In some examples, being below the first threshold can correspond to being a lowest latency possible. In additional or alternative embodiments, determining the resource to be used to handle the AF includes determining the resource based on the resource being capable of keeping the lowest latency possible associated with handling another AF (e.g., a NRT AF) and keeping a latency associated with handling the AF (e.g., a RT AF) below a second threshold.
[0088] In additional or alternative embodiments, determining the resource includes determining the resource to ensure an end-to-end runtime of each AF of a plurality of AFs handled by the multinode cloud-based system are below respective threshold values.
[0089] At block 650, processing circuitry 902 transmits, via communication interface 906, instructions indicating the resource to be used to handle the AF. [0090] Various operations from the flow chart of FIG. 6 may be optional with respect to some embodiments of network nodes and related methods. For example, operations of block 650 may be optional.
[0091] FIG. 7 shows an example of a communication system 700 in accordance with some embodiments.
[0092] In the example, the communication system 700 includes a telecommunication network 702 that includes an access network 704, such as a radio access network (RAN), and a core network 706, which includes one or more core network nodes 708. The access network 704 includes one or more access network nodes, such as network nodes 710a and 710b (one or more of which may be generally referred to as network nodes 710), or any other similar 3rd Generation Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 710 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 712a, 712b, 712c, and 712d (one or more of which may be generally referred to as UEs 712) to the core network 706 over one or more wireless connections.
[0093] Example wireless communications over a wireless connection include transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 700 may include any number of wired or wireless networks, network nodes, UEs, and/or any other components or systems that may facilitate or participate in the communication of data and/or signals whether via wired or wireless connections. The communication system 700 may include and/or interface with any type of communication, telecommunication, data, cellular, radio network, and/or other similar type of system.
[0094] The UEs 712 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and/or operable to communicate wirelessly with the network nodes 710 and other communication devices. Similarly, the network nodes 710 are arranged, capable, configured, and/or operable to communicate directly or indirectly with the UEs 712 and/or with other network nodes or equipment in the telecommunication network 702 to enable and/or provide network access, such as wireless network access, and/or to perform other functions, such as administration in the telecommunication network 702.
[0095] In the depicted example, the core network 706 connects the network nodes 710 to one or more hosts, such as host 716. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 706 includes one more core network nodes (e.g., core network node 708) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and/or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 708. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and/or a User Plane Function (UPF).
[0096] The host 716 may be under the ownership or control of a service provider other than an operator or provider of the access network 704 and/or the telecommunication network 702, and may be operated by the service provider or on behalf of the service provider. The host 716 may host a variety of applications to provide one or more service. Examples of such applications include live and prerecorded audio/video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.
[0097] As a whole, the communication system 700 of FIG. 7 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and/or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and/or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.
[0098] In some examples, the telecommunication network 702 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 702 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 702. For example, the telecommunications network 702 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and/or Massive Machine Type Communication (mMTC)/Massive loT services to yet further UEs.
[0099] In some examples, the UEs 712 are configured to transmit and/or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 704 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 704. Additionally, a UE may be configured for operating in single- or multi- RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).
[0100] In the example, the hub 714 communicates with the access network 704 to facilitate indirect communication between one or more UEs (e.g., UE 712c and/or 712d) and network nodes (e.g., network node 710b). In some examples, the hub 714 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 714 may be a broadband router enabling access to the core network 706 for the UEs. As another example, the hub 714 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 710, or by executable code, script, process, or other instructions in the hub 714. As another example, the hub 714 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 714 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 714 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 714 then provides to the UE either directly, after performing local processing, and/or after adding additional local content. In still another example, the hub 714 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.
[0101] The hub 714 may have a constant/persistent or intermittent connection to the network node 710b. The hub 714 may also allow for a different communication scheme and/or schedule between the hub 714 and UEs (e.g., UE 712c and/or 712d), and between the hub 714 and the core network 706. In other examples, the hub 714 is connected to the core network 706 and/or one or more UEs via a wired connection. Moreover, the hub 714 may be configured to connect to an M2M service provider over the access network 704 and/or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 710 while still connected via the hub 714 via a wired or wireless connection. In some embodiments, the hub 714 may be a dedicated hub - that is, a hub whose primary function is to route communications to/from the UEs from/to the network node 710b. In other embodiments, the hub 714 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 710b, but which is additionally capable of operating as a communication start and/or end point for certain data channels.
[0102] FIG. 8 shows a UE 800 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and/or operable to communicate wirelessly with network nodes and/or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle-mounted or vehicle embedded/integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-loT) UE, a machine type communication (MTC) UE, and/or an enhanced MTC (eMTC) UE.
[0103] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and/or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).
[0104] The UE 800 includes processing circuitry 802 that is operatively coupled via a bus 804 to an input/output interface 806, a power source 808, a memory 810, a communication interface 812, and/or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in FIG. 8. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.
[0105] The processing circuitry 802 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored as machine-readable computer programs in the memory 810. The processing circuitry 802 may be implemented as one or more hardware- implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general- purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 802 may include multiple central processing units (CPUs). [0106] In the example, the input/output interface 806 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and/or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 800. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.
[0107] In some embodiments, the power source 808 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 808 may further include power circuitry for delivering power from the power source 808 itself, and/or an external power source, to the various parts of the UE 800 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 808. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 808 to make the power suitable for the respective components of the UE 800 to which power is supplied.
[0108] The memory 810 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 810 includes one or more application programs 814, such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 816. The memory 810 may store, for use by the UE 800, any of a variety of various operating systems or combinations of operating systems.
[0109] The memory 810 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and/or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUlCC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 810 may allow the UE 800 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 810, which may be or comprise a device-readable storage medium.
[0110] The processing circuitry 802 may be configured to communicate with an access network or other network using the communication interface 812. The communication interface 812 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 822. The communication interface 812 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a network node in an access network). Each transceiver may include a transmitter 818 and/or a receiver 820 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 818 and receiver 820 may be coupled to one or more antennas (e.g., antenna 822) and may share circuit components, software or firmware, or alternatively be implemented separately.
[0111] In the illustrated embodiment, communication functions of the communication interface 812 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communications such as Bluetooth, nearfield communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may be implemented in according to one or more communication protocols and/or standards, such as IEEE 802.11 , Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol/internet protocol (TCP/IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.
[0112] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 812, via a wireless connection to a network node. Data captured by sensors of a UE can be communicated through a wireless connection to a network node via another UE. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).
[0113] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.
[0114] A UE, when in the form of an Internet of Things (loT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city wearable technology, extended industrial application and healthcare. Non-limiting examples of such an loT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door/window sensor, a flood/moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item-tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an loT device comprises circuitry and/or software in dependence of the intended application of the loT device in addition to other components as described in relation to the UE 800 shown in FIG. 8.
[0115] As yet another specific example, in an loT scenario, a UE may represent a machine or other device that performs monitoring and/or measurements, and transmits the results of such monitoring and/or measurements to another UE and/or a network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-loT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and/or reporting on its operational status or other functions associated with its operation.
[0116] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and/or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.
[0117] FIG. 9 shows a network node 900 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and/or operable to communicate directly or indirectly with a UE and/or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)).
[0118] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units and/or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).
[0119] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell/multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, SelfOrganizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and/or Minimization of Drive Tests (MDTs).
[0120] The network node 900 includes a processing circuitry 902, a memory 904, a communication interface 906, and a power source 908. The network node 900 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 900 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 900 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 904 for different RATs) and some components may be reused (e.g., a same antenna 910 may be shared by different RATs). The network node 900 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 900, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 900. [0121] The processing circuitry 902 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and/or encoded logic operable to provide, either alone or in conjunction with other network node 900 components, such as the memory 904, to provide network node 900 functionality.
[0122] In some embodiments, the processing circuitry 902 includes a system on a chip (SOC). In some embodiments, the processing circuitry 902 includes one or more of radio frequency (RF) transceiver circuitry 912 and baseband processing circuitry 914. In some embodiments, the radio frequency (RF) transceiver circuitry 912 and the baseband processing circuitry 914 may be on separate chips (or sets of chips), boards, or units, such as radio units and digital units. In alternative embodiments, part or all of RF transceiver circuitry 912 and baseband processing circuitry 914 may be on the same chip or set of chips, boards, or units.
[0123] The memory 904 may comprise any form of volatile or non-volatile computer-readable memory including, without limitation, persistent storage, solid- state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and/or any other volatile or nonvolatile, non-transitory device-readable and/or computer-executable memory devices that store information, data, and/or instructions that may be used by the processing circuitry 902. The memory 904 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and/or other instructions capable of being executed by the processing circuitry 902 and utilized by the network node 900. The memory 904 may be used to store any calculations made by the processing circuitry 902 and/or any data received via the communication interface 906. In some embodiments, the processing circuitry 902 and memory 904 is integrated.
[0124] The communication interface 906 is used in wired or wireless communication of signaling and/or data between a network node, access network, and/or UE. As illustrated, the communication interface 906 comprises port(s)/terminal(s) 916 to send and receive data, for example to and from a network over a wired connection. The communication interface 906 also includes radio frontend circuitry 918 that may be coupled to, or in certain embodiments a part of, the antenna 910. Radio front-end circuitry 918 comprises filters 920 and amplifiers 922. The radio front-end circuitry 918 may be connected to an antenna 910 and processing circuitry 902. The radio front-end circuitry may be configured to condition signals communicated between antenna 910 and processing circuitry 902. The radio front-end circuitry 918 may receive digital data that is to be sent out to other network nodes or UEs via a wireless connection. The radio front-end circuitry 918 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 920 and/or amplifiers 922. The radio signal may then be transmitted via the antenna 910. Similarly, when receiving data, the antenna 910 may collect radio signals which are then converted into digital data by the radio front-end circuitry 918. The digital data may be passed to the processing circuitry 902. In other embodiments, the communication interface may comprise different components and/or different combinations of components.
[0125] In certain alternative embodiments, the network node 900 does not include separate radio front-end circuitry 918, instead, the processing circuitry 902 includes radio front-end circuitry and is connected to the antenna 910. Similarly, in some embodiments, all or some of the RF transceiver circuitry 912 is part of the communication interface 906. In still other embodiments, the communication interface 906 includes one or more ports or terminals 916, the radio front-end circuitry 918, and the RF transceiver circuitry 912, as part of a radio unit (not shown), and the communication interface 906 communicates with the baseband processing circuitry 914, which is part of a digital unit (not shown).
[0126] The antenna 910 may include one or more antennas, or antenna arrays, configured to send and/or receive wireless signals. The antenna 910 may be coupled to the radio front-end circuitry 918 and may be any type of antenna capable of transmitting and receiving data and/or signals wirelessly. In certain embodiments, the antenna 910 is separate from the network node 900 and connectable to the network node 900 through an interface or port.
[0127] The antenna 910, communication interface 906, and/or the processing circuitry 902 may be configured to perform any receiving operations and/or certain obtaining operations described herein as being performed by the network node. Any information, data and/or signals may be received from a UE, another network node and/or any other network equipment. Similarly, the antenna 910, the communication interface 906, and/or the processing circuitry 902 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and/or signals may be transmitted to a UE, another network node and/or any other network equipment.
[0128] The power source 908 provides power to the various components of network node 900 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 908 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 900 with power for performing the functionality described herein. For example, the network node 900 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 908. As a further example, the power source 908 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.
[0129] Embodiments of the network node 900 may include additional components beyond those shown in FIG. 9 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and/or any functionality necessary to support the subject matter described herein. For example, the network node 900 may include user interface equipment to allow input of information into the network node 900 and to allow output of information from the network node 900. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 900.
[0130] FIG. 10 is a block diagram of a host 1000, which may be an embodiment of the host 716 of FIG. 7, in accordance with various aspects described herein. As used herein, the host 1000 may be or comprise various combinations hardware and/or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1000 may provide one or more services to one or more UEs. [0131] The host 1000 includes processing circuitry 1002 that is operatively coupled via a bus 1004 to an input/output interface 1006, a network interface 1008, a power source 1010, and a memory 1012. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as FIGS. 8 and 9, such that the descriptions thereof are generally applicable to the corresponding components of host 1000.
[0132] The memory 1012 may include one or more computer programs including one or more host application programs 1014 and data 1016, which may include user data, e.g., data generated by a UE for the host 1000 or data generated by the host 1000 for a UE. Embodiments of the host 1000 may utilize only a subset or all of the components shown. The host application programs 1014 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1014 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1000 may select and/or indicate a different host for over-the-top services for a UE. The host application programs 1014 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.
[0133] FIG. 11 is a block diagram illustrating a virtualization environment 1100 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1100 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized.
[0134] Applications 1102 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment Q400 to implement some of the features, functions, and/or benefits of some of the embodiments disclosed herein.
[0135] Hardware 1104 includes processing circuitry, memory that stores software and/or instructions executable by hardware processing circuitry, and/or other hardware devices as described herein, such as a network interface, input/output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1106 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1108a and 1108b (one or more of which may be generally referred to as VMs 1108), and/or perform any of the functions, features and/or benefits described in relation with some embodiments described herein. The virtualization layer 1106 may present a virtual operating platform that appears like networking hardware to the VMs 1108.
[0136] The VMs 1108 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1106. Different embodiments of the instance of a virtual appliance 1102 may be implemented on one or more of VMs 1108, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.
[0137] In the context of NFV, a VM 1108 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, nonvirtualized machine. Each of the VMs 1108, and that part of hardware 1104 that executes that VM, be it hardware dedicated to that VM and/or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1108 on top of the hardware 1104 and corresponds to the application 1102.
[0138] Hardware 1104 may be implemented in a standalone network node with generic or specific components. Hardware 1104 may implement some functions via virtualization. Alternatively, hardware 1104 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 1110, which, among others, oversees lifecycle management of applications 1102. In some embodiments, hardware 1104 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1112 which may alternatively be used for communication between hardware nodes and radio units.
[0139] FIG. 12 shows a communication diagram of a host 1202 communicating via a network node 1204 with a UE 1206 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 712a of FIG. 7 and/or UE 800 of FIG. 8), network node (such as network node 710a of FIG. 7 and/or network node 900 of FIG. 9), and host (such as host 716 of FIG. 7 and/or host 1000 of FIG. 10) discussed in the preceding paragraphs will now be described with reference to FIG. 12.
[0140] Like host 1000, embodiments of host 1202 include hardware, such as a communication interface, processing circuitry, and memory. The host 1202 also includes software, which is stored in or accessible by the host 1202 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1206 connecting via an over-the-top (OTT) connection 1250 extending between the UE 1206 and host 1202. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1250.
[0141] The network node 1204 includes hardware enabling it to communicate with the host 1202 and UE 1206. The connection 1260 may be direct or pass through a core network (like core network 706 of FIG. 7) and/or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.
[0142] The UE 1206 includes hardware and software, which is stored in or accessible by UE 1206 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1206 with the support of the host 1202. In the host 1202, an executing host application may communicate with the executing client application via the OTT connection 1250 terminating at the UE 1206 and host 1202. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1250 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1250.
[0143] The OTT connection 1250 may extend via a connection 1260 between the host 1202 and the network node 1204 and via a wireless connection 1270 between the network node 1204 and the UE 1206 to provide the connection between the host 1202 and the UE 1206. The connection 1260 and wireless connection 1270, over which the OTT connection 1250 may be provided, have been drawn abstractly to illustrate the communication between the host 1202 and the UE 1206 via the network node 1204, without explicit reference to any intermediary devices and the precise routing of messages via these devices.
[0144] As an example of transmitting data via the OTT connection 1250, in step 1208, the host 1202 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1206. In other embodiments, the user data is associated with a UE 1206 that shares data with the host 1202 without explicit human interaction. In step 1210, the host 1202 initiates a transmission carrying the user data towards the UE 1206. The host 1202 may initiate the transmission responsive to a request transmitted by the UE 1206. The request may be caused by human interaction with the UE 1206 or by operation of the client application executing on the UE 1206. The transmission may pass via the network node 1204, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1212, the network node 1204 transmits to the UE 1206 the user data that was carried in the transmission that the host 1202 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1214, the UE 1206 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1206 associated with the host application executed by the host 1202.
[0145] In some examples, the UE 1206 executes a client application which provides user data to the host 1202. The user data may be provided in reaction or response to the data received from the host 1202. Accordingly, in step 1216, the UE 1206 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input/output interface of the UE 1206. Regardless of the specific manner in which the user data was provided, the UE 1206 initiates, in step 1218, transmission of the user data towards the host 1202 via the network node 1204. In step 1220, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1204 receives user data from the UE 1206 and initiates transmission of the received user data towards the host 1202. In step 1222, the host 1202 receives the user data carried in the transmission initiated by the UE 1206.
[0146] One or more of the various embodiments improve the performance of OTT services provided to the UE 1206 using the OTT connection 1250, in which the wireless connection 1270 forms the last segment. More precisely, the teachings of these embodiments may allow a multinode cloud-based system (e.g., a FaaS system) to schedule functions based on requirements of the function and a status of the cloud-based system, and thereby ensure E2E RT runtimes for RT functions. [0147] In an example scenario, factory status information may be collected and analyzed by the host 1202. As another example, the host 1202 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1202 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1202 may store surveillance video uploaded by a UE. As another example, the host 1202 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1202 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and/or transmitting data.
[0148] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1250 between the host 1202 and UE 1206, in response to variations in the measurement results. The measurement procedure and/or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1202 and/or UE 1206. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1250 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1250 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1204. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1202. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 1250 while monitoring propagation times, errors, etc.
[0149] Although the computing devices described herein (e.g., UEs, network nodes, hosts) may include the illustrated combination of hardware components, other embodiments may comprise computing devices with different combinations of components. It is to be understood that these computing devices may comprise any suitable combination of hardware and/or software needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network node, and/or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and/or the functionality of the components may be partitioned between the processing circuitry and the communication interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware. [0150] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and/or by end users and a wireless network generally.

Claims

39 CLAIMS What is claimed is:
1 . A method performed by a network node of a multinode cloud-based system, the method comprising: receiving (610) a deployment request for an application function, AF; determining (620) requirements of the AF including a latency threshold associated with the AF; determining (630) information associated with the multinode cloud-based system; and determining (640) a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloud-based system.
2. The method of Claim 1 , wherein the AF comprises a real-time, RT, AF, wherein determining the requirements of the AF comprises determining a latency threshold associated with the RT AF, and wherein determining the resource comprises determining the resource based on the resource being capable of keeping a latency associated with handling the RT AF below the latency threshold.
3. The method of Claim 1 , wherein the AF comprises a non real-time, NRT, AF, and wherein determining the resource to be used to handle the NRT AF comprises determining the resource based on the resource being capable of keeping a latency associated with handling the NRT AF below a first threshold and keeping a latency associated with handling a real-time, RT, AF handled by the resource below a second threshold.
4. The method of any of Claims 1 -3, wherein receiving the deployment request comprises receiving the deployment request from a remote device via an AF deployment interface of the multinode cloud-based system. 40
5. The method of any of Claims 1 -4, wherein determining the information associated with the multinode cloud-based system comprises receiving an indication of the information from a cluster monitor of the multinode cloud-based system.
6. The method of any of Claims 1 -5, wherein the information comprises at least one of: processing latency, network latency, and service latency associated with each resource of a plurality of resources of the multinode cloud-based system.
7. The method of any of Claims 1 -6, further comprising: transmitting (650) instructions to a deployment controller of the multinode cloud-based system indicating the resource to be used to handle the AF.
8. The method of any of Claims 1 -7, wherein determining the resource comprises determining the resource to ensure an end-to-end runtime of each AF of a plurality of AFs handled by the multinode cloud-based system are below respective threshold values.
9. The method of any of Claims 1 -8, wherein the multinode cloud-based system comprises a function-as-a-service, FaaS, system, and wherein the network node comprises a spatial function scheduler configured to assign real-time, RT, AF and non real-time, NRT, AF to nodes of the multinode cloud based system.
10. A network node (900) operating in a multinode cloud-based system, the network node comprising: processing circuitry (902); and memory (904) coupled to the processing circuitry and having instructions stored therein that are executable by the processing circuitry to cause the network node to perform operations comprising: receiving (610) a deployment request for an application function, AF; determining (620) requirements of the AF including a latency threshold associated with the AF; determining (630) information associated with the multinode cloud-based system; and 41 determining (640) a resource of the multinode cloud-based system to be used to handle the AF based on the requirements of the AF and the information associated with the multinode cloud-based system.
11 . The network node of Claim 10, wherein the AF comprises a real-time, RT, AF, wherein determining the requirements of the AF comprises determining a latency threshold associated with the RT AF, and wherein determining the resource comprises determining the resource based on the resource being capable of keeping a latency associated with handling the RT AF below the latency threshold.
12. The network node of Claim 10, wherein the AF comprises a non real-time, NRT, AF, and wherein determining the resource to be used to handle the NRT AF comprises determining the resource based on the resource being capable of keeping a latency associated with handling the NRT AF below a first threshold and keeping a latency associated with handling a real-time, RT, AF handled by the resource below a second threshold.
13. The network node of any of Claims 10-12, wherein receiving the deployment request comprises receiving the deployment request from a remote device via an AF deployment interface of the multinode cloud-based system.
14. The network node of any of Claims 10-13, wherein determining the information associated with the multinode cloud-based system comprises receiving an indication of the information from a cluster monitor of the multinode cloud-based system.
15. The network node of any of Claims 10-14, wherein the information comprises at least one of: processing latency, network latency, and service latency associated with each resource of a plurality of resources of the multinode cloud-based system.
16. The network node of any of Claims 10-15, the operations further comprising: transmitting (650) instructions to a deployment controller of the multinode cloud-based system indicating the resource to be used to handle the AF.
17. The network node of any of Claims 10-16, wherein determining the resource comprises determining the resource to ensure an end-to-end runtime of each AF of a plurality of AFs handled by the multinode cloud-based system are below respective threshold values.
18. The network node of any of Claims 10-17, wherein the multinode cloud-based system comprises a function-as-a-service, FaaS, system, and wherein the network node comprises a spatial function scheduler configured to assign real-time, RT, AF and non real-time, NRT, AF to nodes of the multinode cloud based system.
19. A computer program comprising program code to be executed by processing circuitry (902) of a network node (900) operating in a multinode cloud-based system, whereby execution of the program code causes the network node to perform operations comprising any operations of Claims 1-9.
20. A computer program product comprising a non-transitory storage medium (904) including program code to be executed by processing circuitry (902) of a network node (900) operating in a multinode cloud-based system, whereby execution of the program code causes the network node to perform operations comprising any operations of Claims 1-9.
21 . A non-transitory computer-readable medium having instructions stored therein that are executable by processing circuitry (902) of a network node (900) operating in a multinode cloud-based system to cause the network node to perform operations comprising any of the operations of Claims 1-9.
PCT/IB2022/058527 2021-09-09 2022-09-09 Real-time function-as-a-service system WO2023037319A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202163242418P 2021-09-09 2021-09-09
US63/242,418 2021-09-09

Publications (1)

Publication Number Publication Date
WO2023037319A1 true WO2023037319A1 (en) 2023-03-16

Family

ID=83689251

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2022/058527 WO2023037319A1 (en) 2021-09-09 2022-09-09 Real-time function-as-a-service system

Country Status (1)

Country Link
WO (1) WO2023037319A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200167196A1 (en) * 2019-04-30 2020-05-28 Intel Corporation Methods and apparatus to execute a workload in an edge environment
US20200296155A1 (en) * 2020-03-27 2020-09-17 Intel Corporation Method, system and product to implement deterministic on-boarding and scheduling of virtualized workloads for edge computing

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200167196A1 (en) * 2019-04-30 2020-05-28 Intel Corporation Methods and apparatus to execute a workload in an edge environment
US20200296155A1 (en) * 2020-03-27 2020-09-17 Intel Corporation Method, system and product to implement deterministic on-boarding and scheduling of virtualized workloads for edge computing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SZALAY MARK ET AL: "Real-time task scheduling in a FaaS cloud", 2021 IEEE 14TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (CLOUD), IEEE, 5 September 2021 (2021-09-05), pages 497 - 507, XP034012976, DOI: 10.1109/CLOUD53861.2021.00065 *

Similar Documents

Publication Publication Date Title
WO2023203240A1 (en) Network slicing fixed wireless access (fwa) use case
WO2023105073A1 (en) Inter-network-node admission control for a sidelink relay ue
WO2023037319A1 (en) Real-time function-as-a-service system
WO2023168657A1 (en) Method and apparatus for selecting lag port for ip flow
WO2023141826A1 (en) Methods and apparatuses for scheduling terminal device
WO2024028838A1 (en) Network power saving in split ng-ran
WO2024102058A1 (en) Scheduling indication of a pdu set prioritization in uplink
WO2024033480A1 (en) Physical data channel scheduling
WO2023084277A1 (en) Machine learning assisted user prioritization method for asynchronous resource allocation problems
WO2023146463A1 (en) Uplink mac scheduling signaling in a communication network
WO2024075129A1 (en) Handling sequential agents in a cognitive framework
WO2024033821A1 (en) Multi-slot transmission with a preconfigured allocation
WO2023186724A1 (en) Radio access network (ran) analytics exposure mechanism
WO2023101580A1 (en) Systems and methods for user equipment assisted buffer size in multi-connectivity
WO2023218383A1 (en) Systems and methods for enabling per service configuration for mt-sdt
WO2023239280A1 (en) Mobile terminated small data transmission - ul response selection
WO2023068983A1 (en) Measurement reporting based on measurement configurations using frequency specific priority indications
WO2024023347A1 (en) Frame number correction for discontinuous reception
WO2023146462A1 (en) Uplink mac scheduling in a communication network
WO2023153991A1 (en) Per data radio bearer (drb) delay threshold configuration
WO2023104353A1 (en) Configurable support for generic virtualized infrastructure manager resources
WO2023209557A1 (en) Intent based automation for predictive route performance
WO2023080829A1 (en) User equipment and inter-ue coordination method perfomed therein
WO2023153978A1 (en) Ue history information (uhi) and mobility history information (mhi) during scg activation/deactivation
WO2023209566A1 (en) Handling of random access partitions and priorities

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22786838

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE