JP2024513628A - RIC SDK - Google Patents

Abstract

To provide a low-latency Near-RT RIC, some embodiments separate the functionality of the RIC into several different components that run on different machines (e.g., running on VMs or pods) that run on the same or different host computers. Some embodiments also provide high-speed interfaces between these machines. Some or all of these interfaces operate in a non-blocking, lockless manner to ensure that critical operations of the Near-RT RIC (e.g., datapath processing) are not delayed due to multiple requests that stall one or more components. Furthermore, each of these RIC components also has an internal architecture designed to operate in a non-blocking manner so that a process of one component cannot block the operation of another process of the component. All of these low-latency features enable the Near-RT RIC to act as a high-speed IO between the E2 nodes and the xApps.

Description

In telecommunications networks, radio access networks (RANs) perform more functions with each iteration of telecommunications standards. That is, 5G RAN performs various additional functions to enable the benefits of 5G compared to traditional standards.

These RAN functions are often performed at base stations (eg, cell towers) where computing power may be limited because they are located between the user devices and the core network.

Some embodiments provide a new RAN intelligent controller (RIC) for telecommunications networks. For example, to provide low-latency near-RT RIC, some embodiments provide RIC functions running on different machines (e.g., VMs or pods) running on the same host computer or different host computers. ) into several different components. Also, some embodiments provide a high speed interface between these machines. Some or all of these interfaces are non-blocking to ensure that critical operations of the near RT RIC (e.g. datapath processing) are not delayed due to multiple requests stalling one or more components. It operates in a non-blocking and lockless manner. Additionally, each of these RIC components also has an internal architecture designed to operate in a non-blocking manner so that one process of the component cannot block the operation of another process of the component. All of these low-latency features enable the near RT RIC to service high-speed IO between base station nodes (ie, E2 nodes) and control plane applications (eg, xApps).

In some embodiments, the near RT RIC includes a datapath pod, a service pod, and an SDL (shared data layer) pod. Part of RIC's low-latency architecture is due to the use of different pods to implement data IO, services, and SDL operations. As such, different resource allocation and management operations can be provided to each of these pods based on their respective needs for the operations they perform. Also, in some embodiments, the RIC provides low latency messaging between its various pods.

Service pods, in some embodiments, perform application (eg, xApp) onboarding, registration, FCAPS (fault, configuration, accounting, performance, security), and other services. It also provides services (such as metric collection, policy provisioning and configuration) to other RIC components. The SDL pod implements the shared data layer of the near RT RIC. The SDL pod of some embodiments also executes one or more service containers to perform one or more pre-processing or post-processing services on data stored in the SDL.

Datapath pods perform data message transfer between base station components of a communications network and perform control and edge applications for this network. In some embodiments, some or all of the datapath services of a datapath pod are incorporated into the datapath thread and control thread of the datapath pod. In other embodiments, data path services are incorporated into a data IO thread, multiple data processing threads (DPTs), and a control thread.

The control thread in some embodiments is the interface to the service pod and the SDL pod for the datapath thread, while in other embodiments it is the interface to the service pod for the datapath thread (datapath (because threads can communicate directly with SDL pods). The control thread in either of these approaches performs the slow control-related operations of the datapath, while one or more datapath threads performs the fast IO operations of the datapath. The control thread in some embodiments has an interface with the service pod to receive configuration data to configure its own operation as well as the operation of the datapath thread.

Embodiments that separate the data path thread into a data IO thread and multiple data processing threads further optimize data IO by pushing the more computationally intensive operations of the data path thread to multiple data path processing threads, This allows for less computationally intensive operations to be performed on the data IO thread. Both of these optimizations are intended to ensure high-speed data path IO (without introducing unnecessary delays), which allows the near RT RIC to connect between base station nodes and control and edge applications. can function as a high-speed interface.

The foregoing Summary of the Invention is intended to serve as a brief introduction to some embodiments of the invention. This is not meant to be an introduction or summary of all inventive subject matter disclosed in this document. The following detailed description and the drawings referred to in the detailed description further explain the embodiments described in the Summary of the Invention as well as other embodiments. Therefore, in order to understand all the embodiments described in this document, a thorough review of the Summary of the Invention, Detailed Description, and Drawings is necessary. Furthermore, since the claimed subject matter may be embodied in other specific forms without departing from the concept of the subject matter, the subject matter may be incorporated into the Summary of the Invention, the Detailed Description, and the Drawings. It is intended not to be limited by the details illustrated in the appended claims, but rather to be defined by the appended claims.

The novel features of the invention are set forth in the appended claims. However, for illustrative purposes, some embodiments of the invention are set forth in the following figures.

1 illustrates an example O-RAN architecture according to some embodiments.

3 illustrates details of components of both a non-real-time RIC and a near-real-time RIC according to some embodiments.

FIG. 3 shows a more detailed diagram of the MAC control assistant of some embodiments.

FIG. 4 shows a more detailed diagram of a user-level tracer of some embodiments.

FIG. 2 shows another diagram of the O-RAN architecture of some embodiments and shows a more detailed diagram of the near real-time RIC.

2 illustrates the deployment of the RIC SDK on a machine running a control plane application in some embodiments.

5 illustrates that some embodiments deploy several RICs to run on several host computers to implement a distributed near RT RIC that includes the RIC components shown in FIGS. 5 and 6. .

Figure 3 shows a RIC running on one host computer with two machines running two control plane applications.

Two RICs are shown running on two host computers, along with two machines running two control plane applications and two RIC SDKs.

1 shows a RIC connecting two control plane applications running on two machines running on a first host computer and running on two other host computers.

A RIC running on a first host computer and connecting two control plane applications running on two machines, one running on the first host computer and one running on another host. Indicates that it runs on a computer.

3 illustrates examples of different standard APIs supported by the distributed near RT RIC platform of some embodiments.

An embodiment is shown in which the SDL cache is part of each RIC SDK running on the same machine as its control plane.

An example of a control or edge application that has pass-through access to a host computer's hardware accelerator to perform some or all of its operations is shown.

3 illustrates processing performed in some embodiments in response to an O-RAN component instructing a CP or edge application to perform an operation that requires the application to use a host computer's hardware accelerator;

An application is shown that performs operations based on data from an E2 node.

2 illustrates another example of a control or edge application having pass-through access to a host computer's hardware accelerator to perform some (or all) of its operations.

FIG. 12 illustrates yet another example of a CP or edge application having pass-through access to a host computer's hardware accelerator to perform some or all of its operations.

Some embodiments illustrate a process used to deploy O-RAN applications with direct pass-through access to a host computer's hardware accelerator.

3 illustrates an example of a CP or edge application having pass-through access to a virtual hardware accelerator defined by a hypervisor running on a host computer.

Figure 3 shows an example of a near RT RIC with several components running on several different machines.

, 22 illustrates a different example for deploying components near the near RT RIC of FIG. 21;

, Another example of a near RT RIC is shown.

An example of a RIC datapath pod is shown.

3 illustrates the processing that a datapath thread performs in some embodiments to process a subscription request from an xApp.

Figure 3 illustrates the processing that a data IO thread and DPT perform in some embodiments to process data messages from an E2 node that are to be received by one or more xApps.

FIG. 4 illustrates the processing that the data IO thread and DPT perform, in some embodiments, to process data messages from an xApp to be sent to an E2 node.

FIG. 4 illustrates an example of a process used by a data IO thread to assign an E2 node to a DPT in some embodiments.

Figure 3 shows a distributed near RT RIC implemented by an active RIC and a standby RIC.

In some embodiments, interfaces between a near RT RIC and an E2 node, and between a near RT RIC and an xApp pod are shown.

Figure 3 shows the E2AP message processing of the near RT RIC datapath pod.

3 illustrates a RIC instance of some embodiments with an SDL pod.

1 conceptually illustrates an electronic system in which some embodiments of the invention are implemented;

The following detailed description of the invention describes and explains numerous details, examples, and embodiments of the invention. However, it will be obvious and clear to those skilled in the art that the invention is not limited to the described embodiments and that the invention may be practiced without some of the specific details and examples described. Will.

Today, there is a movement to push radio access networks (RANs) of telecommunication networks (eg, cellular networks) implemented as O-RANs toward standards that allow interoperability of RAN elements and interfaces. FIG. 1 illustrates an example O-RAN architecture 100, according to some embodiments. The O-RAN architecture 100 includes a non-real-time RIC 105, a near-real-time RAN intelligent controller 115, an open control plane centralized unit (O-CU-CP) 120, an open user plane centralized unit (O-CU-UP) 125, an open distributed unit ( O-DU) 130, an open radio unit (O-RU) 135, and an O-Cloud 140. The O-CU-CP 120, O-CU-UP 125, and O-DU 130 may hereinafter be collectively referred to as managed functions 120-130.

As defined in the standard, the SMO 110 of some embodiments provides integration that allows the SMO to connect to and manage the RIC 115, managed functions 120-130, and O-Cloud 140 via an open interface 150. Contains fabric. Unlike these elements, O-RU 135, in some embodiments, is not managed by SMO 110 and is instead managed by O-DU 130, as indicated by dashed line 160. In some embodiments, O-RU 135 processes and transmits radio frequencies to O-DU 130.

In some embodiments, the managed functions 120-130 are logical nodes that each host a set of protocols. In accordance with the O-RAN standard, for example, the O-CU-CP 120 in some embodiments includes protocols such as Radio Resource Control (RRC) and the control plane portion of the Packet Data Convergence Protocol (PDCP), and the O-CU-UP 125 includes protocols such as the Service Data Adaptation Protocol (SDAP) and the user plane portion of the Packet Data Convergence Protocol (PDCP).

Each of the two RICs is tailored to specific control loop and latency requirements. The near real-time RIC 115 provides programmatic control of open centralized units (O-CU) and open distributed units (O-DU) with time cycles of 10 ms to 1 second. On the other hand, a non-real-time RIC (non-RT RIC) 105 provides higher layer policies that can be implemented in the RAN via a direct connection to a near RT RIC or RAN node. Non-RT RICs are used for control loops longer than 1 second. Each RIC 105 or 115 functions as a platform for executing RAN control applications. These applications may be developed by third party suppliers different from the RIC vendor. These applications are called "xApps" (for near RT RICs 115) and "rApps" (for non-RT RICs).

Near real-time RIC 115, in some embodiments, is a logical aggregation of several functions that utilize data collection and communication via interface 155 to control managed functions 120-130. In some embodiments, non-real-time RIC 105 uses machine learning and model training to manage and optimize managed functions 120-130. The near RT RIC in some of these embodiments also uses machine learning.

In some embodiments, O-Cloud 140 is responsible for creating and hosting virtual network functions (VNFs) used by RIC 115 and managed functions 120-130. In some embodiments, the DU includes a RAN scheduler that is responsible for making per-slot decisions for user scheduling and provides MAC control assistance and user-level tracing. To increase the computing power available in the cloud (i.e., compared to base stations that typically perform RAN functions), the RIC may be implemented in one or more public and/or private cloud data centers. , implements an improved cloud-defined RAN scheduler in the cloud, thereby offloading these MAC control assist and user-level tracing functions from the DU to the RIC. Interface 155 in some embodiments allows the RAN to provide input to functions at the RIC and, in at least some embodiments, receive output computed by these functions at the RIC.

FIG. 2 shows a detailed diagram of the components of both non-real-time RIC 201 and near-real-time RIC 202. Each of the RICs 201 and 202 includes a respective set of analysis functions 210 and 212 and a respective set of optimization functions 214 and 216, each of which is shown in dashed lines to indicate that it is an existing component. . In addition to these existing components, near real-time optimization function 216 includes two new components, MAC Control Assister 220 and User Level Tracer 222, which are shown in solid lines to visually distinguish them from the existing components. In some embodiments, these components are part of a larger MIMO component (eg, along with a MU-MIMO UE pairer and precoder).

In some embodiments, the MAC control assistant 220 performs (1) user equipment (UE) specific beamforming weight calculations based on UL SRS channel signal reception, (2) UE radio frequency (RF) condition prediction, and (3) Various features may be included, such as multi-user, multi-input, multi-output (MU-MIMO) pairing suggestions for MAC schedulers based on UE-specific beams. For each of these functions, some embodiments expose a reporting interface (which provides input data for the DU to RIC function) and a control interface (which provides output data for the RIC to DU function). .

User level tracer 222, in some embodiments, generates L1/L2/L3 level information related to user configuration and traffic performance. This trace data can be used as input to various control algorithms such as MAC schedulers, parameter settings, etc. The user level tracer 222 includes (i) tracing operations that can track user operations within a cell, (ii) tracking user RF conditions, and (iii) different layers (MAC, Radio Link Control (RLC), packet data convergence). (iv) tracking of user data traffic performance within the protocol (PDCP); and (iv) user RF resource consumption.

FIG. 3 shows a more detailed diagram of the MAC control assistant 300 of some embodiments. As described, the MAC control assistant 300 includes a UE-specific beamforming weight calculator (BFWC) 310, a UE RF condition predictor 320, and a MU-MIMO pairing suggester 330. UE-specific BFWC 310 in some embodiments is based on UL SRS channel signal reception. In some embodiments, the MU-MIMO pairing suggester 330 is for a UE-specific beam-based MAC scheduler.

Each of the components 310-330 of MAC control assistant 300 includes an uplink and a downlink, as described. For UE-specific BWC functions, some embodiments include an uplink voice reference signal (UL SRS) channel response matrix that is an input to a weight calculation function and a control interface for a UE-specific beamforming weight matrix. Publish a reporting interface for For the UE RF condition predictor functionality, some embodiments include a reporting interface for downlink (DL) channel condition reports that are inputs to the RF condition predictor and a predicted DL channel condition report for the next scheduling window. Expose control interfaces for channel conditions (including, for example, DL SINR, PMI, and rank). For the MU-MIMO pairing proposal function, some embodiments provide a reporting interface for the UE-specific beamforming weight matrix that is input to the UE pairing proposal function, and a reporting interface for the UE pairing proposal and SINR impact evaluation. Expose the control interface.

FIG. 4 shows a more detailed diagram of the user level tracer 400 of some embodiments. Tracer 400, in some embodiments, includes multiple uplinks 410 and multiple downlinks 415 for performing tracking operations. These operations generate L1/L2/L3 level information related to user settings and traffic performance. This trace data can be used as input to various control algorithms such as MAC schedulers, parameter settings, etc. These tracking operations include: 1) tracking user behavior within the cell, 2) tracking user RF conditions, 3) tracking user data traffic performance at different layers (MAC, RLC, PDCP), and 4) user RF resource consumption. can be tracked.

For these tracing operations, some embodiments expose reporting interfaces for DUs and/or CUs to provide various metrics for user-level tracking operations. These metrics may include selected RRC messages, MAC/RLC/PDCP traffic volume and performance, RF conditions, and RF resource consumption. In some embodiments, messages through these interfaces to the RIC are triggered based on user behavior and/or periodic reporting (eg, for traffic performance and RF conditions/resource consumption).

Tracking operations may track the various user data indicated above and provide this information back to the RAN or to other control algorithms (eg, other algorithms operating on the RIC). For example, these algorithms perform analysis on user data performance from user-level trace operations, determine that certain performance is insufficient, and modify how the RAN is handling user traffic. . Examples of control algorithms that can benefit from user-level tracing in some embodiments are (1) traffic steering, (2) quality of service (QoS) scheduling optimization, (3) user configuration adjustment, and ( 4) Including user operation abnormality detection.

All operations described in Figures 3-4 (i.e., MAC scheduler functions and user-level tracking operations) can be performed without excessive delay, due to the increased computational power available on the RIC in the cloud. Calculations become possible. For example, some or all of these operations may be performed in the RIC using machine learning (eg, using a machine-trained network).

5 illustrates another diagram of an O-RAN architecture of some embodiments, showing a more detailed view of a near-real-time RIC. The architecture 500 includes a SMO 505 having a non-real-time RIC 510, a distributed near-real-time RIC 515, and an E2 node 520 (e.g., an O-DU and/or O-CU node). The distributed near-real-time RIC 515 includes a messaging infrastructure 540, a set of services (e.g., 550, 552, 554, and 556), a shared data layer 560, a database 570, and a set of termination interfaces (e.g., 580, 582, and 584). As shown, a set of embedded apps (e.g., 530, 532, 534) use this distributed near-RT RIC. As described further below, in some embodiments, the distributed near-RT RIC 515 is realized by multiple RICs running on multiple host computers.

As shown, the set of services includes a conflict migration service 550, an app subscription management service 552, a management service 554, and a security service 556. Additionally, the set of termination interfaces includes an O1 termination interface 580 that connects the SMO to the near real-time RIC, an A1 termination interface 582 that connects the non-real-time RIC to the near real-time RIC, and an E2 termination interface 584 that connects the E2 node to the near real-time RIC. include. Each of the apps, in some embodiments, represents a different function of the RIC that uses data sent from the E2 node 520. For example, app 530 corresponds to UE-specific BFWC 310 of MAC control assistant 300, app 532 corresponds to UE RF condition predictor 320 of MAC control assistant 300, and so on.

In some embodiments, the purpose of framework 500 is to offload compute-intensive near real-time functions and return results to O-DU (e.g., provided to E2 node 520 via an E2 interface). do). In some embodiments, the results can be used to assist or enhance real-time decisions at the MAC layer. Three example use cases of the MAC control assist framework, each example specific to a different component of the MAC control assister (e.g., UE-specific BFWC, UE RF condition predictor, MU-MIMO pairing suggester), and one of the user-level tracers. Two example use cases are described below.

A first example use case is specific to UE-specific beamforming weight calculations based on the UL SRS signal reception component of the MAC control assist framework (eg, component 310 of MAC control assister 300). In some embodiments of this use case, the input metrics may include options based on UL SRS, such as raw SRS received data, and SRS channel response matrices from channel estimation.

In some embodiments, the algorithm for generating the output metric evaluates the optimal beamforming weights to reach the user. Some embodiments use conventional signal processing algorithms based on channel models. Alternatively, or in combination, machine learning-based algorithms that utilize raw data input are used, which require feedback from the DU within E2 node 520.

In some embodiments, the output metric obtained from the algorithm includes a beamforming weight matrix for the user. In some embodiments, the BFW can also be mapped to a beam index from a pre-designed beam set. The DU in some embodiments uses matrices to control MIMO antenna array gain/phasing within the RU (eg, O-RU 135 in architecture 100) for transmitting and receiving user data.

A second use case example is specific to the UE RF Condition Predictor component of the MAC Control Assist framework (eg, component 320 of MAC Control Assister 300). In this second use case, according to some embodiments, the input metrics are at least from the UE, such as wideband or subband CQI/PMI/RI for DL or SRS for UL. Includes channel reports. The input metrics of some embodiments may also choose to include assistance information such as UE distance, UE positioning, etc.

In some embodiments, the app algorithm for this second use case is intended to predict the UE's RF conditions based on observations. Some embodiments utilize traditional signal processing algorithms based on channel and mobility models. Alternatively, or in combination, some embodiments also use machine learning-based algorithms that use other factors such as data input and potentially site layout (requiring feedback from the DU). do.

The output metrics for this use case are, in some embodiments, the user's predicted channel conditions for the next scheduling window, as well as the predicted downlink and uplink SINR, precoding matrices (e.g., if applicable). ), and the SU-MIMO layer. In some embodiments, these output metrics are used by the DU for user link adaptation of PDCCH/PDSCH/PUSCH transmissions.

The third use case is specific to MU-MIMO pairing proposals or MAC scheduler components (eg, component 330 of MAC control assistant 300). The input metrics for this example use case, in some embodiments, include at least a UE-specific BFW matrix and a UE RF condition estimate. In addition to the UE-specific BFW matrix and the UE RF condition estimate, some embodiments may also include supporting metrics, such as user data demand, as input metrics.

The app algorithm for this use case, in some embodiments, is intended to identify users that can be paired for MU-MIMO operation. For example, some embodiments of the third use case use traditional signal processing algorithms based on information theory and cross-channel covariance estimation. Alternatively, or in combination, some embodiments use machine learning-based algorithms that use data input, which again requires feedback from the DU.

In some embodiments, the output metrics for this third use case may include a UE pairing proposal and impact assessment for the SINR and SU-MIMO layers. Additionally, the DU of some embodiments uses power metrics to select users for RF scheduling and determine transmission efficiency.

Examples of user-level tracer use cases may include QoS scheduling optimization, with the goal of adjusting users' scheduling priorities of RF resources to optimize quality of service. Input for some embodiments of this use case may include quality of service goals from user subscriptions. In some embodiments, user-level tracing includes (1) tracking user RF conditions, (2) tracking user data traffic performance at different layers (e.g., MAC/RLC/PDCP), and ( 3) Includes tracking user RF resource consumption.

In some embodiments, an app algorithm is based on QoS targets and observed user traffic performance and can be used to determine that a user's resource allocation is insufficient. The algorithmic format may be logic-based or machine learning-based in some embodiments. In some embodiments, the output includes recommendations issued to the MAC scheduler to adjust traffic priorities or link adaptation to improve performance.

On each machine (e.g., each VM or pod) that runs a control plane application, some embodiments implement a RIC SDK between the control plane application on the machine and the set of one or more elements of the RAN. Configure it to function as an interface. In some embodiments, the RIC SDK provides a set of APIs (e.g., a framework) that allow applications to communicate with a distributed near real-time (RT) RIC implemented by two or more near real-time RICs. . Examples of such applications include xApps and other control plane and edge applications, in some embodiments. In O-RAN, xApps perform control plane, monitoring, and data processing operations. The following discussion with respect to FIGS. 6 and 8-20 refers to control plane applications (eg, 615, 815, 820, 915, 920, etc.). These control plane applications, in some embodiments, are xApps in the O-RAN system.

FIG. 6 illustrates the deployment of a RIC SDK 605 on a machine 610 running a control plane application 615 in some embodiments. As described, one or more machines 610 operate on each of a number of host computers 607 within one or more data centers. In some embodiments, the RIC SDK 605 on each machine 610 provides network connectivity to a set of RAN elements (e.g., E2 nodes 520, shared data layer 560, management services 554, SMO 505, etc.) for control plane applications. Contains a set of network connection processes to establish. Using the RIC SDK process eliminates the need for control plane applications on the machine to perform network connectivity operations. In some embodiments, the set of network connection processes in each RIC SDK of each machine establishes and maintains and controls network connections between the machine and the set of RAN elements used by the machine's control plane applications. Handles data packet transmission to and from a set of RAN elements for plain applications.

Control plane applications on each machine communicate with a set of RAN elements via a high-level API 620 that the RIC SDK translates to a low-level API 625. In some embodiments, at least a subset of low-level API calls 625 are specified by a standards organization. Additionally, in some embodiments, the high-level API 620 is written in a high-level programming language (e.g., C++) and the low-level API calls establish and maintain network connections and pass data packets over these connections. Contains low-level calls that pass through.

The set of RAN elements that the RIC SDK connects to the control plane applications on the machine includes RAN elements produced or developed by different RAN vendors and/or developers. These RAN elements, in some embodiments, include the CU 630 and DU 635 of the RAN. The SDK also communicates with the CU and DU via an E2 interface defined by a low-level standard, while the control plane applications on the machine communicate with the CU and DU via the RIC SDK using high-level API calls. In some embodiments, the high-level API calls define the E2 interface operations at a high-level application layer that does not include low-level transport or network operations.

In combination or alternatively, the set of RAN elements with which the RIC SDK connects with the control plane application 615 on its machine 610 includes the RIC's network elements. Again, these network elements in some embodiments include RAN elements produced and/or developed by different RAN vendors and/or developers. These RIC elements in some embodiments include a shared data layer 560, data path input/output (I/O) elements, and application and management services 552 and 554 in some embodiments. FIG. 7 shows how some embodiments operate on several host computers to implement a distributed near RT RIC 700 that includes the RIC components shown in FIGS. 5 and 6. Indicates that you want to deploy. In some embodiments, one RIC 705 runs on each host computer that also runs control plane applications 615. In other embodiments, control plane application 615 may run on a host computer that does not run the RIC. For example, in some embodiments, one or more control plane applications run on one or more host computers that have a graphics processing unit (GPU), but the RIC does not require GPU processing power; , does not work on such host computer.

The RIC SDK also connects the control plane application to other control plane applications not running on other machines via the distributed near RT RIC. In other words, the RIC SDK and distributed near RT RIC in some embodiments serve as a communication interface between control plane applications. In some embodiments, different control plane applications are developed by different application developers who use a common set of RIC APIs to communicate with each other via a distributed near RT RIC. In some of these embodiments, the distributed near RT RIC adds one or more parameters to the API call when forwarding the API call from one control application to another.

Figures 8-11 illustrate example RIC architectures in which a RIC SDK and a distributed near RT RIC establish a communication interface between control plane applications. These architectures are mutually exclusive in some embodiments, while in other embodiments a combination of two or more of these architectures is used. FIG. 8 shows a RIC 800 running on one host computer 805 with two machines 810 and 812 running two control plane applications 815 and 820. RIC 800 receives API calls from CP application 815 via RIC SDKs 802 and 804 running on machines 810 and 812, forwards the API calls to CP application 820, and sends responses to these API calls to a second from the first CP application 820 to the first CP application 815. It also passes API calls from the second CP application 820 to the first CP application 815 and responds from the first CP application 815 to the second CP application 820 .

Figure 9 shows two machines 910 and 912 with two RICs 900 and 901 running on two host computers 905 and 907 on which two control plane applications 915 and 920 and two RIC SDKs 902 and 904 run. As shown, API calls from the first CP application 915 to the second CP application 920 are forwarded through the first RIC SDK 902, the first RIC 900, the second RIC 901, and the second RIC SDK 904. The second CP application's responses to these API calls to the first CP application 915 traverse the reverse path from the second RIC SDK 904, the second RIC 901, the first RIC 900, and the first RIC SDK 902.

API calls from the second CP application 920 to the first CP application 915 are transferred via the second RIC SDK 904, second RIC 901, first RIC 900, and first RIC SDK 902; A response from the first CP application 915 to the second CP application 920 for the API call is transferred via the first RIC SDK 902, the first RIC 900, the second RIC 901, and the second RIC SDK 904. .

FIG. 10 shows a system running on a first host computer 1005 to connect two control plane applications 1015 and 1020 running on two machines 1010 and 1012 running on two other host computers 1006 and 1007. RIC1000 is shown. RIC 1000 receives API calls from CP application 1015 via RIC SDKs 1002 and 1004 running on machines 1010 and 1012, forwards the API calls to CP application 1020, and sends responses to these API calls to a second Passed from CP application 1020 to first CP application 1015. Additionally, the API call is passed from the second CP application 1020 to the first CP application 1015, and the first CP application 1015 responds to the second CP application 1020.

FIG. 11 shows a first host computer for connecting two control plane applications 1115 and 1120 running on two machines 1110 and 1112, one running on host computer 1105 and one running on host computer 1106. RIC 1100 is shown running on RIC 1105. RIC 1100 receives API calls from CP application 1115 via RIC SDKs 1102 and 1104 running on machines 1110 and 1112, forwards the API calls to CP application 1120, and sends responses to these API calls to a second from the first CP application 1120 to the first CP application 1115. Through these SDKs 1102 and 1104, the RIC 1100 also passes API calls from the second CP application 1120 to the first CP application 1115 and responds from the first CP application 1115 to the second CP application 1120.

FIG. 12 shows examples of different standard APIs supported by the distributed near RT RIC platform of some embodiments. As described, the distributed near RT RIC platform 1200 in some embodiments uses the E2, O1, and A1 interfaces defined by the O-RAN standards body. The E2 API is used to communicate with E2 O-RAN nodes such as O-CU-CP 1202, O-CU-UP 1204, O-DU 1206. It also communicates with the non-real-time RIC platform 1208 using the A1 API and with the SMO 1210 using the O1 API.

For each of the time E2, A1, and O1 APIs, the RIC SDK 1215 provides control plane applications 1220 for communicating with the E2 nodes 1202-1206, non-real-time RIC platform 1208, and SMO 1210 using the RIC SDK and the distributed near RT RIC platform. Provide high-level compatible API. In FIG. 12, the upper level corresponding APIs of E2, O1, and Al interfaces are designated using prime signs as E2' API call, O1' API call, and AE API call. These high-level compatible APIs are APIs that are not defined by standards bodies, but which the RIC SDK and/or distributed near RT RIC converts into standard API calls.

FIG. 12 also shows that the control plane applications 1220 communicate with each other via the RIC SDK and the distributed near RT RIC and communicate with one or more elements of the distributed near RT RIC (e.g., shared data layer 560, data path input/output (I/ O) illustrates some internal RIC APIs to enable communication with elements and application and management services 552 and 554).

The enablement API is an API used in some embodiments to enable control plane applications 1220 to communicate with each other. As described above with reference to FIGS. 8-11, in some embodiments these APIs pass through a distributed near RT RIC. In other embodiments, these APIs allow the control plane application's RIC SDKs to communicate directly with each other without going through other components of the distributed near RT RIC. To this end, FIG. 12 includes dashed bidirectional arrows between the RIC SDKs 1215 of two control plane applications 1220 to indicate that, in some embodiments, the RIC SDKs 1215 of these applications communicate directly with each other.

Enablement APIs in some embodiments include registration APIs, service discovery APIs, and inter-app communication APIs. The registration API allows applications 1220 (e.g., xApps) to provide their network identifiers (e.g., their network address and available L4 ports) and provide their functionality (e.g., perform channel prediction). It is used to introduce itself to other applications 1220 by doing so. The service discovery API allows a control plane application 1220 (e.g., an xApp) to query a service directory (e.g., a distributed near RT RIC) for other control plane applications (e.g., other xApps) that provide a particular service. make it possible. Inter-app communication APIs allow control plane applications to communicate with each other to pass data or request specific actions.

Some embodiments deploy an SDL cache on the same host computer as a control plane application and use the cache to service at least a subset of the control plane application's SDL storage access requests. In some embodiments, the control plane application and SDL cache run on a machine that runs on the host computer. In other embodiments, the SDL cache runs on the same host computer, but outside of the machine on which the control plane application runs. In some of these embodiments, multiple control plane applications running on the same host computer use a common SDL cache on that host computer.

The SDL cache, in some embodiments, is part of the RIC running on the same host computer as the control plane. In other embodiments, the SDL cache is part of the RIC SDK running on the same machine as the control plane application. In any of these embodiments, the RIC or RIC SDK synchronization process synchronizes data stored in the SDL cache with data stored in SDL storage.

In some embodiments, the SDL storage runs on a different host computer than the host computer on which the control plane application runs, and in other embodiments, at least a portion of the SDL storage runs on a host computer on which the control plane application runs. Run on the same host computer. Additionally, in some embodiments, the RIC or RIC SDK forwards the SDL access request from the control plane application to the SDL storage if the RIC SDK is unable to process the SDL access request via the SDL cache. For example, if the SDL cache does not store data requested by a control plane application, the RIC or RIC SDK cannot service SDL access requests through the SDL cache.

FIG. 13 shows an embodiment in which the SDL cache 1302 is part of each RIC SDK 1300 running on the same machine 1305 as its control plane application 1310. As described, the RIC SDK 1300 synchronizes the query manager 132 that processes SDL requests from the CP application 1310 and the data stored in the SDL cache with the data stored in the SDL storage 1350 of the SDL 1355 of the distributed near RT RIC 1330. and a synchronization service 1327. In this example, SDL storage 1350 runs on a different host computer than the host computer on which control plane application 1310 runs. However, in other embodiments, at least a portion of SDL storage 1350 runs on the same host computer that control plane application 1310 runs.

When a control plane application 1310 uses a high-level API call to read or write data to SDL storage, the RIC SDK 1300's query manager 1325 first checks whether the data record being read or written is in the SDL cache 1302. Determine whether it is stored. In that case, query manager 1325 reads from or writes to this record. If the operation is a write operation, synchronization service 1327 writes new data to SDL storage 1350 in real time or on a batch basis. On the other hand, if the query manager 1325 of the RIC SDK 1300 determines that the data record being read or written is not stored in the SDL cache 1302, it queries the SDL layer of the distributed near RT RIC to perform the requested read or write operation. Pass the API call to. Upon passing this API call, the RIC SDK 1300 modifies the format of this call and/or modifies the parameters provided with this call in some embodiments.

Some embodiments include various methods for offloading operations within an Open Radio Access Network (RAN) onto a control plane (CP) or with hardware accelerators within a Software Defined Data Center (SDDC). Provide edge applications that run on host computers. For example, in a CP or edge application running on a machine running on a host computer with a hardware accelerator, the method of some embodiments receives data from an O-RAN E2 unit on which the operation must be performed. do. This method uses the machine's drivers to communicate directly with the hardware accelerator and instruct it to perform a set of computations related to the operation. This driver allows communication with the hardware accelerator to bypass the intermediate set of drivers running on the host computer between the machine's drivers and the hardware accelerator. Through this driver, the application of some embodiments receives the calculation result, which in turn is transmitted to one or more O-RAN components (e.g., the E2 unit that provided the data, another E2 unit, or another xApp).

14-20 illustrate several different embodiments for offloading O-RAN operations to a CP or edge application with pass-through access to a host computer's hardware accelerator. Examples of such hardware accelerators include graphical processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and structured ASICs.

FIG. 14 shows an example of a CP or edge application 1402 having pass-through access to a hardware accelerator 1450 of a host computer 1410 to perform some or all of its operations. As described, each application 1402 runs on a pod 1404 that has an accelerator driver 1412 that has direct pass-through access to the accelerator 1450 of the host computer 1410. Each pod 1404 operates within a VM 1406 (ie, runs on a VM 1406) running on a hypervisor 1408 of the host computer.

In some embodiments, pods are small deployable computing units that can be created and managed in Kubernetes. A pod includes a group of one or more containers with shared storage and network resources, and specifications for how the containers should run. In some embodiments, the contents of a pod are always co-located, scheduled at the same time, and run in a shared context. A pod models an application-specific logical host computer. A pod includes one or more application containers that communicate with each other. In some embodiments, a pod's shared context is a collection of operating system namespaces (eg, Linux cgroups). Within the context of a pod, further sub-isolation may apply to individual applications.

Each pod's accelerator driver 1412 has direct access to the hardware accelerator 1450, which bypasses the VM 1406 and hypervisor 1408 hardware accelerator drivers 1414 and 1416. In some embodiments, hypervisor 1408 runs on the operating system (not shown) of host computer 1410. In these embodiments, each pod's accelerator driver 1412's direct access to the hardware accelerator 1450 also bypasses the operating system's hardware accelerator driver.

To communicate with the hardware accelerator, each application 1402 in some embodiments communicates via a RIC SDK 1430 running on its pod. For example, in some embodiments, each application 1402 communicates with the hardware accelerator 1450 using high-level APIs of the RIC SDK 1430. RIC SDK 1430 then converts the high-level API to the low-level API needed to communicate with the machine's driver 1412, which relays the communication to the hardware accelerator 1450. The low-level API is provided by a first company associated with selling the hardware accelerator 1450, while the RIC SDK 1430 is provided by a second company associated with selling the RIC SDK 1430. In some embodiments, the low-level API used by RIC SDK 1430 is an API specified in API library 1432 associated with hardware accelerator 1450.

FIG. 15 illustrates a process 1500 implementing the methods of some embodiments. Process 1500 is executed in response to an O-RAN component instructing a CP or edge application to perform an operation that requires the application to use its host computer's hardware accelerator. This process 1500 is described below with reference to FIG. 16, which depicts an application 1402 performing operations based on data received from an E2 node 1650.

As shown in FIG. 15, process 1500 begins (at 1505) when application 1402 receives data from O-RAN E2 unit 1650 running on host computer 1610. In some embodiments, application 1402 subscribes to data from E2 unit 1650, and the data received at 1505 is responsive to this subscription. This subscription is done via a distributed near RT RIC in some embodiments. Host computers 1410 and 1610 of application 1402 and E2 unit 1650 operate in one SDDC in some embodiments. In other embodiments, these two host computers 1410 and 1610 operate at two different physical locations. For example, host computer 1410 operates at a first location while host computer 1610 operates at a second location proximate to the O-RAN cell site. In some embodiments, the second location does not include a computer with a hardware accelerator to perform complex operations, including the accepted operation.

Application 1402 (1) receives data from E2 unit 1650 (at 1505) via distributed near RT RIC 1680 formed of near RT RICs 1640 and 1645 running on host computers 1410 and 1610; and (2) Receives the RIC SDK 1430 running on that pod 1404. Application 1402 then uses hardware accelerator 1450 to perform a set of computations associated with the operation (at 1510).

To communicate with hardware accelerator 1450, application 1402 uses high-level APIs provided by RIC SDK 1430. RIC SDK 1430 then converts the high-level API to a low-level API defined in an API library 1432 associated with hardware accelerator 1450. These low-level APIs are then communicated to the hardware accelerator 1450 by the pod's driver 1412 via direct pass-through access to the accelerator 1450, bypassing the VM 1406 and hypervisor 1408 drivers 1414 and 1416. Through this driver 1412, the API defined in the API library 1432, and the RIC SDK 1430, the application 1402 also receives the results of operations (eg, calculations) performed by the hardware accelerator 1450.

The application 1402 provides the results of its operations (at 1515) to one or more O-RAN components, such as the E2 unit 1650 that provided the data or SDL storage that initiated the process 1500. This result is provided via the RIC SDK 1430 and the distributed near RT RIC 1680. In other embodiments, the application 1402 sends the results of its operations (via the RIC SDK 1430) to another host computer running on the same host computer or on a different host computer as the application performing the operations using hardware accelerators 1450. O-RAN E2 unit or one or more other applications running on the machine (other than the E2 unit that provided the data on which the application performed the operation). Process 1500 ends after 1515.

In other embodiments, pass-through access of O-RAN control or edge applications is used in other deployment settings. For example, FIG. 17 shows another example of CP or edge applications 1702 that have pass-through access to a hardware accelerator 1750 of a host computer 1710 to perform some (or all) of their operations. In this example, each application 1702 (1) runs on a pod 1704 that runs on a VM 1706 and (2) uses an accelerator driver 1712 of this VM 1706 that has direct pass-through access to the accelerator 1750 of the host computer 1710. The VM 1706 runs on a hypervisor 1708 that runs on the host computer 1710. The virtual machine's accelerator driver 1712 bypasses the hardware accelerator driver 1716 of the hypervisor 1708. In some embodiments, the hypervisor 1708 runs on an operating system (not shown) of the host computer 1710. In these embodiments, the virtual machine's accelerator driver 1712's direct access to the hardware accelerator 1750 bypasses the operating system's hardware accelerator driver.

To use the hardware accelerator 1750, each application 1702 in some embodiments uses the high-level API of the RIC SDK 1730 (running on its pod 1704) to communicate with the hardware accelerator 1750. RIC SDK 1730 translates the high-level API to the low-level API needed to communicate with the VM's driver 1712 and then relays the communication to the hardware accelerator 1750. In some embodiments, the low-level API used by RIC SDK 1730 is an API defined in API library 1732 associated with hardware accelerator 1750. This API library 1732 is part of the VM 1706 driver interface.

FIG. 18 shows yet another example of a CP or edge application 1802 having pass-through access to a hardware accelerator 1850 of a host computer 1810 to perform some or all of its operations. In this example, each application 1802 (1) runs on a VM 1804 running on a hypervisor 1806 running on a host computer 1810 and (2) has direct pass-through access to an accelerator 1850 on the host computer 1810. The accelerator driver 1812 of the VM 1804 is used.

The virtual machine's accelerator driver 1812 bypasses the hypervisor's 1806 hardware accelerator driver 1816. In some embodiments, hypervisor 1806 runs on the operating system (not shown) of host computer 1810. In these embodiments, the virtual machine's accelerator driver 1812's direct access to the hardware accelerator 1850 bypasses the operating system's hardware accelerator driver.

To use the hardware accelerator 1850, each application 1802 of some embodiments uses the high-level APIs of the RIC SDK 1730 (running on its pod 1804) to communicate with the hardware accelerator 1850. The RIC SDK 1830 translates the high-level APIs into the low-level APIs required to communicate with the VM's driver 1812, and then relays the communication to the hardware accelerator 1850. In some embodiments, the low-level APIs used by the RIC SDK 1830 are the APIs specified in an API library 1832 associated with the hardware accelerator 1850. This API library 1832 is part of the driver interface of the VM 1806.

Those of ordinary skill in the art will recognize that pass-through access for O-RAN control or edge applications is used in other deployment configurations in other embodiments. For example, instead of running on pods, applications in other embodiments run on containers. These embodiments then use their pod or VM's hardware accelerator driver to have pass-through access to the hardware accelerator for control or edge applications. In some of these embodiments, the control or edge application communicates with the hardware accelerator through its associated RIC SDK, communicates with other O-RAN components, and communicates with the application through its associated RIC SDK and O-RAN components. communicates with other O-RAN components (to receive data and provide the results of its data processing) through distributed near RT RICs that connect the O-RAN components. In some embodiments, the control or edge application of these embodiments performs a process similar to process 1500 of FIG. 15.

Direct pass-through access to the hardware accelerators described above is highly beneficial for O-RAN. RIC is all about decoupling the intelligence previously embedded in the RAN software (CU and DU) and moving it to the cloud. One of the benefits of this is to use more advanced computing in the cloud for xApp and edge operations (ML, deep learning, reinforcement learning for control algorithms, etc.). DUs close to cell sites have very high network caps, X, making it economically unfeasible to place a GPU at each cell site and typically cannot perform precomputation.

By using hardware accelerators (GPUs, FPGAs, eASICs, ASICs) in the SDDC, some embodiments execute complex control algorithms in the cloud. Examples of such xApps include massive MIMO beamforming and multi-user (MU) MIMO user pairing, as discussed above. In general, xApps that can benefit from massive parallelism can benefit from GPUs or other accelerators. The use of ASICs is beneficial for channel decoding/encoding (Turbo coding, LDPC coding, etc.). In some embodiments, the RIC is typically on the same worker VM as the xApps. However, in other embodiments, the RIC runs on a different host computer so that more xApps that require GPUs and other hardware accelerators can run on hosts that have GPUs and/or other hardware accelerators.

FIG. 19 illustrates a process that some embodiments use to deploy an O-RAN application with direct pass-through access to a host computer's hardware accelerator. To install an application on a host computer, process 1900 selects (at 1905) a set of one or more installation files that include descriptions for configuring the application's pass-through access to hardware accelerators on the host computer. . In some embodiments, the set of files includes one description file that specifies direct pass-through access for the application to the computer's hardware accelerator.

Process 1900 causes a program running on the host computer to pass calls from a particular hardware accelerator driver associated with the application to the hardware accelerator based on the description regarding pass-through access. The configured set of installation files is used (at 1910) without the intervening set of one or more drivers operating on the host computer between the accelerator and the accelerator. This configuration allows an application to bypass a set of intervening drivers when instructing a hardware accelerator to perform application operations and receiving results of the operations from the hardware accelerator.

In some embodiments, the program configured at 1910 is a host operating system, and in other embodiments a hypervisor running on a host computer. In yet another embodiment, the program is a virtual machine (VM) and the application runs on a pod or a container running on the VM. The process 1900 processes the remaining set of installation files selected at 1905 to complete the installation of the application (at 1915) and ends. In other embodiments, the process 1900 performs the configuration of the program as its last operation at 1910 rather than as its first operation. In yet another embodiment, it performs this configuration as one of its intervening installation operations.

Before performing selection and configuration, the deployment process of some embodiments identifies a host computer from a number of host computers as the computer on which the application is to be installed. The process of some embodiments includes determining that the application requires a hardware accelerator, identifying a set of host computers each configuring the hardware accelerator, and selecting a host computer from the set of host computers. identifies the host computer. This process includes (1) determining that an application needs to communicate with a set of one or more other applications running on a selected host computer; and (2) a set of other applications running concurrently on a selected host computer. Select the host computer by selecting the host computer as a set of other applications. In this manner, installing an application along with a set of other applications on a selected host computer reduces communication delays between the application and the other set of applications.

Some embodiments have O-RAN control or edge application hardware accelerator drivers and virtualization provided by an intervening virtualization application (e.g., a hypervisor) running on the same host computer as the application. communicate with hardware accelerators. For example, the method of some embodiments deploys a virtualized application on a host computer to share the resources of the host computer among several machines running on the host computer. The computer has a first set of one or more physical hardware accelerators.

This method deploys some applications on several machines and performs some O-RAN related operations on a set of O-RAN components. Via the virtualization application, the method defines a second set of two or more virtual hardware accelerators that are mapped by the virtualization application to the first set of physical hardware accelerators. In this method, different virtual hardware accelerators are assigned to different applications. The method also allows an application to be configured to perform operations using the assigned virtual hardware accelerator.

In some embodiments, the deployed machine is a pod and the application is deployed to run on the pod. At least two pods run on one VM running on top of a virtualized application. This VM includes a hardware accelerator driver configured to communicate with two different virtual hardware accelerators for two applications running on two pods. In other embodiments, multiple pods run on a single VM running on top of a virtualized application, and each pod is configured to communicate with a virtual hardware accelerator assigned to its driver. Has a hardware accelerator driver.

FIG. 20 shows CP or edge applications having pass-through access to virtual hardware accelerators 2052 and 2054 defined by hypervisor 2008 running on host computer 2010 to perform some or all of their operations. An example of 2002 is shown below. As described, each application 2002 runs on a pod 2004 that has an accelerator driver 2012 with direct pass-through access to a virtual accelerator 2052 or 2054. Each pod 2004 runs within a VM 2006 (ie, runs on a VM 2006) running on the host computer's hypervisor 2008.

Each pod's accelerator driver 2012 has direct access to the virtual accelerator 2052 or 2054, which bypasses the VM 2006 and hypervisor 2008 accelerator drivers 2014 and 2016. In some embodiments, hypervisor 2008 runs on the operating system (not shown) of host computer 2010. In these embodiments, each pod's accelerator driver 2012's direct access to the virtual accelerator 2052 or 2054 also bypasses the operating system's hardware accelerator driver.

As described, the virtual accelerators 2052 and 2054 communicate with the hardware accelerator 2050 through an accelerator manager 2060 of the hypervisor 2008. The accelerator manager 2060 allows the virtual accelerators 2052 and 2054 (and their associated applications 2002) to share a single hardware accelerator 2050 and operate with the accelerator 2050 as if it were dedicated to the respective applications and pods 2002 and 2004. Examples of such hardware accelerators 2050 include graphical processing units (GPUs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and structured ASICs.

To communicate with the virtual accelerator 2052 or 2054, each application 2002 in some embodiments communicates via a RIC SDK 2030 running on the pod 2004. For example, in some embodiments, each application 2002 uses high-level APIs of the RIC SDK 2030 to communicate with its virtual accelerator 2052 or 2054. RIC SDK 2030 then translates the high-level API into the low-level API needed to communicate with each machine's driver 2012, which relays the communication to virtual accelerator 2052 or 2054. Virtual accelerator 2052 or 2054 then relays the communication to hardware accelerator 2050 via accelerator manager 2060.

As discussed above with reference to FIG. 14, in some embodiments, the low-level API is provided by a first company associated with the sale of the hardware accelerator 2050, while the RIC SDK 2030 is provided by a first company associated with the sale of the RIC SDK 2030. provided by a second company associated with. In some embodiments, the low-level API used by RIC SDK 2030 is an API defined in API library 2032 associated with hardware accelerator 2050. Each application 2002 receives the results of the operations of the hardware accelerator 2050 via the accelerator manager 2060, its virtual accelerator 2052 or 2054, its driver 2012, and its RIC SDK 2030.

To provide low-latency near RT RIC, some embodiments deploy RIC functionality on different machines (e.g., running on VMs or pods) running on the same host computer or on different host computers. Separate it into several different components that work. Also, some embodiments provide a high speed interface between these machines. Some or all of these interfaces may be non-blocking to ensure that critical operations of the near RT RIC (e.g. datapath processing) are not delayed due to multiple requests stalling one or more components. , operates in a lockless manner. Additionally, each of these RIC components also has an internal architecture designed to operate in a non-blocking manner so that one process of the component cannot block the operation of another process of the component. All these low-latency features allow the near RT RIC to act as a high-speed IO between the E2 node and the xApp.

FIG. 21 shows an example of a near RT RIC 2100 with several components running on several different machines. In this example, the near RT RIC is divided into three pods. These are data path pod 2105, service pod 2110, and SDL pod 2115. In some embodiments, this RIC (1) processes E2AP messages between E2 node 2118 and xApps 2120, (2) manages connections with E2 node 2118, and (3) processes xApps to E2 nodes. (4) handle xApp liveness operations; The RIC 2100 provides reliable, low-latency messaging between various components, between E2 nodes and xApps, and between E2 nodes/xApps and RIC components. Part of RIC's low-latency architecture is due to the use of different pods to implement data IO, services, and SDL operations. As such, different resource allocation and management operations can be provided to each of the different pods based on the respective needs of the operations they perform on the plurality of different pods.

Each of the three RIC pods 2105, 2110, and 2115 communicates with one or more xApp pods 2120. In some embodiments, each pod (2105, 2110, 2115, or 2120) has hardware resources (e.g., CPU, memory, disk storage, network IO, etc.) are allocated. Additionally, in some embodiments, each pod has its own high availability and lifecycle update settings that match each pod's unique needs.

Service pod 2110 performs xApp onboarding, registration, FCAPS (fault, configuration, accounting, performance, security), and other services in some embodiments. For example, in some embodiments, the service pod 2110 provides management services 554 for near RT RICs and performs O1 termination 580 and A1 termination 582 to the SMO and its associated non-RT RICs. In some embodiments, each of these components 554, 580, and 582 runs on a separate container within the service pod 2110, and in other embodiments, two or more of these components It runs on one container in 2110.

As mentioned above, in some embodiments the A1 interface is between the near RT RIC and the non-RT RIC. Through this interface, the near RT RIC relays the relevant network information reported by the E2 nodes (e.g. CU and DU), and the non-RT RIC relays the relevant network information reported by the E2 nodes (e.g. for control use case operation at non-RT granularity). Provide control commands. The O1 interface is between the near RT RIC and SMO and is used in some embodiments for discovery, configuration, resource management, autoscaling, lifecycle management, and fault tolerance.

RIC management services 554 in some embodiments include services provided by the near RT RIC to xApps and other RIC components. Examples of services provided to xApps include an xApp service registry/directory (an xApp can be used to identify other xApps associated with a distributed near RT RIC and the operations performed by these other xApps). ), and FCAP operations (metric collection, policy provisioning, configuration, etc.). In some embodiments, an xApp can query a service registry/directory to identify other xApps or other xApps that perform a particular service, and when an xApp is added to the directory, the xApps and can register to receive notifications about its features.

Examples of FCAP operations performed by the service pod 2110 for xApps include metrics that collect CPU and memory usage for analysis to raise alerts, configuration operations to configure or reconfigure xApps, and metrics from xApps. Accounting operations are included to collect data necessary for accounting and performance operations to collect and analyze data to quantify the performance of the xApp.

For other RIC components (such as datapath pod 2105 and SDL pod 2115), service pod 2110 also performs services such as metric collection, policy provisioning, and configuration. The service pod 2110 can be viewed as a local controller that performs operations in the direction of a central controller, or SMO. Through the SMO, the service pod 2110 receives configurations and policies to distribute to xApps and other RIC components. Also, for SMO, service pod 2110 provides metrics, log and trace data collected from xApps and/or RIC components (eg, data path pods and SDL pods). In some embodiments, service pods can be scaled (e.g., replicated) and backed up independently of other pods. In some embodiments, the service pod has a data cache that is a cache for the SMO's time series database. In this cache, the service pod stores statistics, logs, trace data, and other metrics collected from xApps and one or more RIC components before uploading this data to the SMO database.

SDL pod 2115 implements SDL 560 and its associated database 570. As described further below, the SDL pod 2115 of some embodiments also executes one or more service containers to perform one or more pre-processing or post-processing services on the data stored in the SDL. Execute. Similar to service pods, SDL pods in some embodiments can be scaled (e.g., replicated) and backed up independently of other pods.

Datapath pod 2105 contains several important near RT RIC components. These are E2 termination 584, conflict mitigation 550, application subscription management 552, and RIC SDK interface 2150. As described further below, some or all of these datapath services in some embodiments are incorporated into the datapath thread and control thread of the datapath pod. In other embodiments, data path services are incorporated into a data IO thread, multiple data processing threads, and a control thread.

A thread is a component of a process that runs on a computer. A process can be an application or part of a larger application. A thread is a sequence of programmed instructions that can be managed independently from other threads of the process. Allows multiple threads of a given process to run simultaneously while sharing the memory allocated to the process (for example, by using the multithreading capabilities of a multicore processor). Multithreading is a programming and execution model that allows multiple threads to exist within the context of a single process. These threads share the process's resources but can run independently.

The control thread in some embodiments is the interface to the service pod and the SDL pod for the datapath thread, while in other embodiments it is the interface to the service pod for the datapath thread (datapath (because threads can communicate directly with SDL pods). The control thread in either of these approaches performs the slow control-related operations of the datapath, while one or more datapath threads performs the fast IO operations of the datapath. The control thread in some embodiments has an interface with the service pod to receive configuration data to configure its own operation as well as the operation of the datapath thread.

Embodiments that separate the data path thread into a data IO thread and multiple data processing threads further optimize data IO by pushing the more computationally intensive operations of the data path thread to multiple data path processing threads, which This enables low-computation operations to be performed on the data IO thread. Both of these optimizations are intended to ensure high-speed datapath IO (one that does not introduce unnecessary delays), allowing the near RT RIC to act as a high-speed interface between the E2 node 2118 and the xApps 2120. .

As mentioned above, pods 2105, 2110 and 2115 communicate via a high speed pod interface. In some embodiments, pod-to-pod connections are established via SCTP (Streaming Control Transmission Protocol) or even faster shared memory (shmem) connections. In some embodiments, shared memory connections are used only between pairs of pods running on the same host computer. Examples of such pod pairs are (1) datapath pod and SDL pod, (2) datapath pod and service pod, (3) service pod and SDL pod, (4) xApp pod and datapath pod, (5) There are xApp pods, SDL pods, etc. Shared memory is lockless, and in some embodiments access to it is non-blocking. Other embodiments use low speed interfaces (eg, gRPC) between service pod 2110 and other pods 2105, #115, and 2120 as service pods, but less important than other pods.

In some embodiments, the different pods (e.g., 2105, 2110, and 2115) of the near RT RIC can run on the same host computer, or on more than one host computer. In other embodiments, one or more pods (e.g., service pod 2110) always run on a separate host computer than the other pods (e.g., datapath pod 2105 and SDL pod 2115). Also, in some embodiments, pods 2105, 2110, and 2115 run on one host computer 2205 along with one or more xApp pods 2220a, and other xApp pods 2220b run on other host computers 2210, as shown in FIG. 22. In other embodiments, two pods 2105, 2110, and 2115 run on one host computer with one or more xApp pods, and one pod 2105, 2110, and 2115 runs on another host computer with one or more xApp pods.

For example, FIG. 23 shows a datapath pod 2105 and a service pod 2110 running on host computer 2300 with two xApp pods 2320a, while SDL pod 2115 runs on host computer 2310 with two xApp pods 2320b. In some embodiments, pods that require hardware accelerators are placed on a host computer that has such hardware resources, and pods that do not require these accelerators are placed on a host computer that has such hardware resources, as described above. located on a host computer that does not have In some embodiments, SDL pods and xApp pods use hardware accelerators, and data path pods and service pods do not. Various examples of pods that can benefit from hardware accelerators, and bypassing to these accelerators, are discussed further above and below.

Also, while some near RT RICs are described above and below as being implemented using pods, other embodiments near RT RICs use VMs to implement the RIC components. Additionally, even in embodiments that implement different RIC components in pods, some or all of the pods run on a VM, such as a lightweight VM (eg, Photon VM provided by VMware, Inc.).

In addition to using high-speed communication interfaces between pods, in some embodiments some or all of the pods use non-blocking, lockless communication protocols and architectures. For example, datapath pod 2105 uses non-blocking, lockless communication between the threads and processes that make up the pod. Datapath pod 2105 also uses non-blocking lockless communication when communicating with service pod 2110, SDL pod 2115, and xApp pod 2120. In non-blocking communication, a first component sending requests to a second component does not stop the second component from operating if the second component processes too many requests. In such a case, the second component instructs the first component to resend the request at a later time. Datapath pods use single-threaded processing without thread handoff, and therefore use lockless communication. Therefore, there is no need to lock a portion of memory to prevent another process thread from modifying it for a temporary period of time.

The communication interface between the RIC SDK interface 2150 of the data path pod 2105 and the RIC SDK 2112 of the xApp pod 2120 is also new in some embodiments. In some embodiments, this interface parses the header of the E2AP message received from the E2 node and into a new encapsulation header that encapsulates the E2SM payload of the E2AP message along with some or all of the original E2AP header. Store some or all of the analyzed components. In performing this encapsulation, the SDK interface 2150 of some embodiments reduces message size overhead for communication from one pod to another (e.g., reduces the size of the E2 global ID value, etc.). ) to perform certain optimizations such as efficient data packing. These optimizations improve the efficiency of near RT RIC datapath pod and xApp pod communications.

The near RT RIC in other embodiments has one or more other pods. For example, FIG. 24 shows a near RT RIC 2400 that, in addition to pods 2105, 2110, and 2115, also includes a life cycle management (LCM) pod 2405. The LCM pod is a special service pod that upgrades each of the other pods 2105, 2110, and 2115 in the near RT RIC 2400. Separating lifecycle management from the service pod 2110 allows the service pod 2110 to be more easily upgraded.

In some embodiments, LCM pod 2405 uses different upgrade methods to upgrade different pods. For example, the LCM pod of some embodiments seamlessly migrates from an active data store to another standby data store to replicate SDL data storage and perform hitless upgrades of SDL. On the other hand, upgrading the datapath pod requires a more complex procedure for the LCM pod. This is because the active and standby datapath pods are configured to dual home the E2 node and each xApp, and the active datapath pod is configured to replicate state with the standby datapath.

FIG. 25 shows a more detailed diagram of the near RT RIC 2500 for some embodiments. As described, the near RT RIC 2500 includes a data path pod 2505, a service pod 2510, and an SDL pod 2515. SDL pod 2515 includes RIC 2500's SDL agent 2526 and shared SDL storage 2528, while service pod 2510 includes service agent 2524 along with O1 and A1 termination interfaces 2535 and 2537. Datapath pod 2505 includes a datapath thread 2507 and a control thread 2509.

Datapath thread 2507 provides high speed datapath IO for the near RT RIC between E2 node 2518 and xApps 2520. The dataplane capacity of the RIC in some embodiments can be scaled up by implementing RIC datapath IO with one control thread and multiple datapath threads that share the load for datapath processing for the datapath pods. Some such implementations are further described below. Control thread 2509 performs some control operations related to the RIC's datapath. The near RT RIC 2500 separates the control and datapath threads because dataIO operations need to be fast and should not be slowed down by control operations that can operate at slower speeds. In some embodiments, the control and datapath threads are two threads in a single process (i.e., they run in the same shared memory address space).

In some embodiments, each of these threads may communicate with other components of this architecture (e.g., RIC SDK, Service Pod Agent, SDL Agent, and/or E2 Node) to the extent that each of these threads communicates with these other components. use a non-blocking, lockless interface to communicate with the Also, in some embodiments, both threads use minimal OS system calls and execute as an infinite loop. As described further below, the datapath thread and the control thread are connected to two circular rings 2522 ( data is exchanged via a .cbuf).

Control thread 2509 serves as a control interface to E2 node 2518, SMO 2530 (via service pod agent 2524), xApps (eg, via SCTP), and the SDL layer (via SDL agent 2526). In some embodiments, the control thread is the main thread for communicating with these external entities, while the data path thread in some embodiments is the main thread for communicating with these external entities, as described further below. to communicate with SDL2515.

Control thread 2509 in some embodiments handles all control functions. This thread sends various control parameters to other functions and, in some embodiments, implements admission control. In other embodiments, data path thread 2507 implements admission control and SMO via service pods specifies admission control. In some embodiments, control thread 2509 has control channel communication with the xApp pod's RIC SDK via SCTP. In other embodiments, the control thread communicates with the xApp pod's RIC SDK via gRPC. Also, in some embodiments, the control thread communicates with the RIC SDK via shared memory (shmem) when the xApp pod and data path pod run on the same host computer.

Control thread 2509 also provides a transmission mechanism for transmitting statistics, log and trace data generated by data path thread 2507. In some embodiments, some or all of this data is transferred to SDL pod 2515 via SDL agent 2526 and/or to SMO 2530 via service agent 2524. The control thread 2509 in some embodiments negotiates security keys with the E2 node peers and passes these keys to the data path thread, which uses those keys to perform encryption/decryption operations. do.

Data path thread 2507 provides high speed IO between E2 nodes and xApp. This thread handles the RIC SDK interface and E2 termination operations, and in some embodiments conflict mitigation and xApp subscription operations. This thread is the ASN of the E2AP message. 1. Execute decoding to extract message data. In some embodiments, the datapath thread does not decode the E2SM payload of these messages. Datapath thread 2507 verifies E2 and xApp messages and sequences. In some embodiments, the message types include E2 node setup and service updates, E2 node indication reports, xApp initiation subscriptions for E2 node data, and xApp initiation control requests.

Datapath thread 2507 in some embodiments executes an E2 state machine to create and maintain state (eg, state of E2 nodes, subscriptions to E2 nodes, etc.) on behalf of xApps. Also, in some embodiments, the datapath thread performs table lookups to send messages to xApps requesting data. This thread also handles control requests from xApps to E2 nodes and forwards responses to these requests from E2 nodes to xApps.

The datapath thread communicates with the xApps via SCTP if the xApps are on a separate host computer, or via shared memory if the xApps are on the same host computer. In some embodiments, xApp messages have CRC bits to detect corruption. These messages also carry timestamps and may be compressed in some embodiments. Data path thread 2507 performs data replication for multiple subscriptions. Data path thread 2507 also performs data path security operations, such as by signing, encrypting, and decrypting data messages.

As discussed above and further below, data path thread 2507 communicates with control thread 2509 in some embodiments via a pair of rings 2522. In some embodiments, the frequency of messages between two threads is adjustable (eg, configurable) from sub-milliseconds to seconds per ring pair. Through the control thread, data path thread 2507 receives configuration data updates and state changes. The data path thread 2507 generates statistics, logs and traces and provides the generated statistics, logs and trace data to the control thread for storage within the SDL and/or for providing to the SMO.

Datapath thread 2507 also performs conflict management operations when multiple xApps attempt to set the same parameter on the same E2 node at the same time. For example, conflict management operations ensure that two xApps do not attempt to change mobile network settings (such as antenna orientation) in different ways within a short period of time. In some embodiments, data path thread conflict management employs different methodologies to address different types of conflicts. For example, (1) for one set of requests, reject a second request that changes a parameter after receiving the conflicting first request for some time; and (2) for another set of requests. , reject a request for parameters from one xApp when another high-priority xApp makes a conflicting request for the same parameters; It only accepts requests made by xApps that are allowed to make such requests during that time. Policies for handling these conflicts are provided by the SMO 2530 via the service pod's agent 2524.

In FIG. 25, each xApp pod 2520 can run one or more xApps 2532 and interfaces with a datapath pod 2505, an SDL pod 2515, and a service pod 2510 via a RIC SDK 2534 running on the xApp pod. Each RIC SDK provides a high-level interface for xApps to communicate with the RIC and E2 nodes. This high-level interface hides the underlying implementation details. The RIC SDK communicates with the RIC instances via high-speed data IO communication channels (such as shared memory or SCTP).

The RIC SDK also uses control communication channels with the service pod 2510 and control thread 2509 for xApp control operations such as xApp onboarding, registration, features, subscriptions, FCAPS, etc. In some embodiments, control channel communication between the SDK and the control thread 2509 is via shared memory when the xApp pod (and its SDK) and the data path pod run on the same host computer, and through shared memory when the xApp pod (and its SDK) and datapath pod run on the same host computer. When it works, it is done via SCTP. Also, in some embodiments, the control channel communication between the xApp pod (and its SDK) and the service pod may be through shared memory if the SDK and service pod run on the same host computer, or through shared memory if the SDK and service pod run on the same host computer, If it runs on top of this, it is done via gRPC. Other embodiments use SCTP for communication between the SDK and the service pod when the xApp pod (and its SDK) and the service pod run on different host computers.

In some embodiments, the RIC SDK uses proto bufs when communicating with service pods via gRPC. Additionally, in some embodiments where communication between the RIC SDK and the data path pod occurs via shared memory, the shared memory communication uses protocol buf. The RIC SDK has APIs for data functions such as E2 messages to and from E2 nodes. These APIs include onboarding xApps (name, version, capabilities), message subscriptions, keep-alive messaging, A1 and O1 interface communication with SMO via service pods (e.g., sending statistics, logs, trace data to SMO or It also includes control function messages such as communication for storing in a time series database on the service pod (Prometheus, ELK, etc.).

In some embodiments, the data path thread and control thread are assigned to one processor core, the SDL is assigned to another processor core (to separate them from the data and control threads), and the service pod is assigned to yet another processor core. assign. If one or more xApps run on the same host computer as the RIC, the xApps are assigned to a different core than the RIC pod. In a RIC pod, multiple xApps can be assigned to the same core, or individual cores can be assigned to individual xApps as needed.

To further improve RIC and xApp performance, other embodiments perform other hardware allocation optimizations, such as specific memory allocations (e.g., larger RAM allocations) and specific IO allocations. Examples of dedicated IO allocations for some pods include (1) SRIOV allocations for an xApp pod on one host computer to communicate with a datapath pod on another host computer, (2) SRIOV allocations for datapath pods to communicate with E2 nodes, (3) SRIOV allocations for an xApp pod on one host computer to communicate with a service pod on another host computer, and (4) gRPC or SCTP communication using SRIOV allocations with gRPC communication where gRPC communication has a lower bandwidth allocation and a lower priority than SCTP communication.

In some embodiments, one RIC and several xApps are bundled to run on different pods running on one VM. In some embodiments with different sets of xApps, multiple instances of the RIC may also be deployed. Also, in some embodiments, xApps that need to communicate with each other are bundled on the same VM.

As mentioned above, some embodiments implement the RIC datapath as one data IO thread with multiple datapath processing threads, rather than as one datapath thread. In some embodiments, each data path processing thread (DPT) is responsible for performing data path processing for a different set of E2 nodes, with each E2 node assigned to only one data path processing thread. bear. In some embodiments, the data IO thread hashes the E2 node identifier included in the message and uses the hashed value (obtained by hashing) as the DPT identifier of the DPT that needs to process the data message. Identify the DPT associated with the E2 message or xApp message by using it as an index into the lookup table provided.

FIG. 26 shows an example of a RIC datapath pod 2600 with one data IO thread 2605, one control thread 2610, and multiple DPTs 2615. The DPT shares the datapath processing load of the datapath pod 2600. As described, there are pairs of cbuf rings 2620 between each DPT 2615 and data IO thread 2605, between each DPT 2615 and control thread 2610, and between data IO thread 2605 and control thread 2610. Each ring 2620 of a cbuf pair passes data messages in one direction from one of the two threads associated with the ring to the other, and one ring passes data messages in one direction (e.g., from the first thread to the second one thread to another (e.g., from a second thread to a first thread).

Separating the data IO thread 2605 from multiple DPTs 2615 optimizes data IO for the datapath pod 2600 by pushing more compute-intensive operations to the DPTs, thereby allowing less compute-intensive IO operations to be performed. Enables execution within data IO thread 2605. This optimization ensures high-speed data path IO (without unnecessary delays), allowing the RIC to act as a high-speed interface between the E2 node and the xApp. Also, each E2 node is typically responsible for only one DPT thread 2615, which is responsible for several E2 nodes. Since each E2 node is processed by one specific DPT, two DPTs cannot attempt to modify one or more records associated with one E2 node. Therefore, the data path pod 2600 does not need to lock the record of the E2 node since its responsibilities for communicating with the E2 node are clearly differentiated.

Data IO thread 2605: (1) manages connections to E2 nodes and xApp pods; (2) sends data messages over these connections to and from E2 nodes and xApp pods; (3) (4) control ring communications with control thread 2610 and DPT 2615; and (5) generate statistics, log and trace data regarding messages it processes.

Each DPT thread 2615 performs (1) message decoding and encoding operations (e.g., message encryption and decryption operations), (2) message validation operations, (3) sequence validation operations, (4) E2 node and Maintaining a state machine to track the state of xApp requests and subscriptions; (5) performing conflict management; (6) controlling ring communication with control thread 2610 and DPT 2615; and (7) statistics and logging regarding messages processed. and generate trace data.

FIG. 27 illustrates a process 2700 that a data path thread performs to process a subscription request from an xApp in some embodiments. As shown, processing begins when the DPT receives an xApp subscription request (at 2705) from the data IO thread. A subscription request is directed to a particular E2 node for a particular set of data tuples (eg, a particular set of operating parameters or other parameters) that the particular E2 node maintains.

At 2710, process 2700 determines whether it has already subscribed to a particular E2 node to receive a particular set of data tuples. This means that when the DPT previously sent one or more subscription requests to a particular E2 node, a particular set of data tuples, or a larger data tuple containing a particular set of data tuples, or This is a case of collective request.

If the process 27100 determines (at 2710) that it has already subscribed to a particular E2 node to receive a particular set of data tuples, then (at 2715) to add a new record or update a previously specified record, specifying within this record the particular set of data tuples that the xApp should receive. After 2715, the process ends.

On the other hand, if the process 27100 determines (at 2710) that it has not yet subscribed to a particular E2 node to receive a particular set of data tuples, then it has an active subscription with this node. Send the first subscription to the particular E2 node if not included or does not include all data tuples of the particular data tuple specified in the request received at 2705 If so, send the updated subscription to the node.

Therefore, in such a case, process 2700 (at 2720) adds a new record or updates a previously specified record for the xApp in this E2 node's subscription list, and specifies the particular set of data tuples that the xApp should receive. It then sends the updated subscription request to the particular E2 node using the previously assigned RIC request ID. This updated subscription will receive all the data in a particular set of the particular requested data tuples if none of these tuples were previously requested by a previous subscription to the particular E2 node. Specify a tuple. Or specify some of the other data tuples in a particular set if those data tuples were previously requested by one or more previous subscriptions to a particular E2 node. After 2725, process 2700 ends.

FIG. 28 illustrates a process 2800 that data IO thread 2605 and DPT 2615 perform, in some embodiments, to process data messages from an E2 node that are to be received by one or more xApps. As illustrated, process 2800 begins when a data message is received (at 2805) by a datapath pod via an SCTP connection with an E2 node. At 2810, data IO thread 2605 generates a hash value from the E2 node's ID. The hash value is then used (at 2815) as an index into a lookup table to identify the DPT assigned to process the message associated with the E2 node.

At 2820, the data IO thread passes the received data message to the identified DPT (i.e., the DPT identified at 2815) along a cbuf ring 2620 for passing the message from the data IO thread to the identified DPT. hand over. Next, at 2825, the DPT uses its data structure records (eg, records maintained by its state machine) to identify the set of one or more xApps from which to obtain the E2 message. In some embodiments, the identified set of xApps are xApps that have subscribed to receive data (eg, all data or a subset of data) from the E2 node.

At 2830, the DPT identifies a data message for the data IO thread 2605 to send to the identified set of xApps. This data message is in an encapsulated format with reference to Table 1 below. The DPT then passes the data message to the data IO thread 2605 (at 2835) along the cbuf ring 2620 for passing messages from the DPT 2615 to the data IO thread 2605. Next, at 2840, data IO thread 2605 retrieves the data message from cbuf ring 2620, identifies the xApps that need to receive the data message, and then sends the data message to each identified xApp. After 2840, processing ends.

FIG. 29 shows a process 2900 that data IO thread 2605 and DPT 2615 execute, in some embodiments, to process data messages from xApps to be sent to E2 nodes. As illustrated, process 2900 begins (at 2905) when a data message is received by a datapath pod via an SCTP connection or shared memory communication with the xApp RIC SDK. This message is in an encapsulated format as described below with reference to Table 1. This message includes an E2 node identifier that identifies the E2 node receiving this message.

At 2910, data IO thread 2605 generates a hash value from the E2 node's ID. The hash value is then used (at 2915) as an index into a lookup table to identify the DPT assigned to process the message associated with the E2 node. At 2920, the data IO thread passes the received data message to the identified DPT (i.e., the DPT identified at 2915) along a cbuf ring 2620 for passing the message from the data IO thread to the identified DPT. hand over.

Next, at 2925, the DPT uses its data structure records (eg, records maintained by its state machine) to identify the E2 node that should receive the message. In some embodiments, the data message is a subscription request and the identified E2 node is an E2 node to which the xApp wishes to subscribe. At 2930, the DPT specifies a data message for the data IO thread 2605 to send to the identified E2 node. This data message is in the E2AP message format required by the standard. The DPT then passes the data message to the data IO thread 2605 (at 2935) along the cbuf ring 2620 for passing messages from the DPT 2615 to the data IO thread 2605. Next, at 2940, data IO thread 2605 retrieves the data message from cbuf ring 2620, identifies the E2 nodes that need to receive the data message, and sends the data message to each identified E2 node. After 2940, processing ends.

In some embodiments, DPT 2615 may determine that there is no need to send a new subscription message to the E2 node identified at 2925. For example, before receiving a subscription request for a set of data tuples from the first xApp (at 2905) from the E2 node, the data path pod previously sent to the second xApp or send a subscription request to the same E2 node for a larger set of data tuples that includes the data tuple requested by the first xApp. In such a case, the DPT 2615 can simply add the first xApp to the E2 node's subscription list and then provide the first xApp with the values it subsequently receives from the E2 node. In some embodiments, the DPT 2615 also provides previously received values from the E2 node stored in the SDL to the first xApp, or directs the xApp to obtain these values from the SDL.

In some cases, the first xApp requests additional data tuples from the E2 node that the second xApp did not previously request. In such a case, DPT 2615 prepares an updated subscription message for the data IO thread to send to the E2 node to request the newly requested data tuple by the first xApp. The DPT also prepares such messages when the second xApp requests additional data tuples from the E2 node after the first subscription.

In some embodiments, service pod 2510 configures data path pod 2600 to instantiate N DPTs when it starts, where N is an integer greater than one. For near RT RIC datapath pods 2600, the number N is calculated in some embodiments based on the expected number of E2 nodes and xApps that communicate with the E2 nodes via the near RT RIC. In some embodiments, the data IO thread 2605 of the data path pod 2600 then assigns E2 nodes to the DPT based on the order of subscription requests it receives and the load on the DPT at the time of these requests. .

FIG. 30 illustrates an example of the process used by the data IO thread to assign E2 nodes to DPTs in some embodiments. As described, process 3000 begins when data IO thread 2605 receives (3005) a first subscription request for a particular E2 node from an xApp associated with the data IO thread's near RT RIC. The first subscription request for a particular E2 node means that no other subscription request was previously received for this particular E2 node by the data IO thread.

Next, at 3010, the data IO thread 2605 generates an N-bit hash value from the global E2 node ID of the particular E2 node. Here, N is an integer (eg, 6 or 8). This N-bit value is used to identify a particular E2 node within a hash LUT (lookup table), as described below. At 3015, process 3000 selects a particular DPT for a particular E2 node based on the current load on each DPT of data path pod 2600 (e.g., by selecting the DPT with the least amount of load). do. In some embodiments, the current load is based solely on the number of E2 nodes assigned to each DPT, while in other embodiments the current load is based on the number of E2 nodes and the xApps to those nodes. Based on number of subscriptions. In yet other embodiments, the current load is calculated in other ways.

Next, at 3020, the process 3000 creates a record in the LUT in which it associates the N-bit hash value with the identifier of the particular DPT selected at 3015 for the particular E2 node. In some embodiments, the N-bit hash value is an index into a LUT that identifies a record that specifies the ID of a particular E2 node. At 3020, process 3000 also designates the state of this record as active.

At a subsequent point in time, if the data IO thread encounters a situation where all xApps have canceled their subscriptions to a particular E2 node, process 3000 maintains the LUT record created in 3020, but this record Change the status of to inactive. The data IO thread maintains this inactive state until the next time the xApp sends a subscription request for a particular E2 node. At that point, the state of this record is changed to active again. This status value is used as a mechanism to prevent the data IO thread from having to continually re-assign E2 nodes to DPTs.

FIG. 31 shows a distributed near RT RIC implemented by an active RIC 3102 and a standby RIC 3104. As shown, E2 nodes 3118 and xApp pods 3120 communicate with the active RIC 3102 until one or more components of this RIC fail. If the active RIC fails, the standby RIC 3104 becomes the active RIC and the E2 nodes and xApp pods continue to communicate via the RIC 3104, which is currently the active RIC.

Both of these RICs have the same components: data path pod 3105, service pod 3110, and SDL pod 3115. The datapath pod is shown to include a control thread 3109 and a datapath thread 3107. Instead of one data path thread 3107, some embodiments use one data IO thread and multiple DPTs, as described above. In some embodiments, the active RIC 3102 is implemented by a first set of one or more computers, while the standby RIC 3104 is implemented by a different second set of one or more computers.

As shown, each E2 node 3118 has dual homed SCTP connections with data path threads 3107 of active and standby RICs 3102 and 3104. Similarly, each xApp pod 3120 has dual homed SCTP connections with data path threads 3107 of active and standby RICs 3102 and 3104. Dual homing connectivity is a feature provided by SCTP. When a first component connects to an active/standby pair of components via a dual-homed connection, the first component can automatically switch to using the standby component in the event of a failure of the active component. . Thus, using dual-homed SCTP connections, each E2 node or xApp pod can be switched to the standby RIC 3104's datapath thread 3107 when the active RIC 3102 or its datapath pod fails.

As described, the RIC SDK interface 3122 of the data path thread 3107 of the active RIC 3102 transfers messages it receives from the xApp RIC SDK and messages it sends to the xApp RIC SDK to the RIC SDK interface 3122 of the data path 3107 of the standby RIC 3104. Transfer to. This, in some embodiments, allows the standby RIC's datapath thread 3107 to update its state machine to match the state of the active RIC's datapath thread 3107. Also, as described, the synchronization agents 3127 of the active and standby RICs 3102 and 3104 synchronize the SDL storage 3126 of the standby RIC 3104 with the SDL storage 3126 of the active RIC 3102. All components of active and standby RICs 3102 and 3104 are managed consistently by SMO 3130.

FIG. 32 illustrates the interfaces between a near RT RIC 3200 and an E2 node 3205 and between a near RT RIC 3200 and an xApp pod 3210 in some embodiments. In some embodiments, the near RT RIC is one of the near RT RICs described above. As discussed above and shown in FIG. 32, the near RT RIC in some embodiments uses SCTP interfaces 3220 and 3222 with both E2 nodes 3205 and xApp pods 3210. When the xApp pod and the near RT RIC run on the same host computer, some embodiments use shared memory as an interface between the near RT RIC and the xApp pod, as shown in the figure. These interfaces keep message exchanges fast, minimizing encoding and decoding overhead across all paths, and minimizing latency (eg, one ASN decoding and one ASN encoding). In some embodiments, the interface 3220 between the E2 node and the near RT RIC follows the E2AP specification, and all message exchanges comply with the E2AP specification.

Also, in some embodiments, the near RT RIC to xApp pod interface 3222 uses a novel encapsulation header, described below with reference to Table 1. Interface 3222 handles a mix of different types of messages. Examples of such messages in some embodiments include (1) the entire E2AP message from the E2 node (e.g., an E2 setup request); (2) the entire E2SM content (i.e., the entire E2AP message payload) along with the E2AP header. (3) internal messages between the near RT RIC and the xApp pod (e.g., a message from the near RT RIC that the xApp's previous message caused an error); (4) an internal message from the xApp to the near Contains messages to RT RIC or E2 nodes. In some embodiments, the E2 content may not be ASN1 encoded (eg, a portion of the subscription request may not be encoded).

In some embodiments, the Near RT RIC 3200 may be configured on a case-by-case basis to decode only E2AP messages or to decode the entire E2AP header along with the E2SM payload before sending the message to the xApp. . In some cases, the near RT RIC sends the entire E2AP header, in other cases only a part of this header. In the RIC E2AP message processing of some embodiments, all fields are in network byte order, and the near RT RIC 3200 operates in that order as much as possible. To display fields, some embodiments may transform the data into host order. In some embodiments, the near RT RIC 3200 does not see the E2SM payload, while in other embodiments it does (eg, to avoid duplicate subscription errors).

In some embodiments, the RAN capability ID is E2 node specific. xApps do not subscribe to RAN functionality between E2 nodes since each subscription belongs to an individual E2 node. Also, in some embodiments, the RIC request ID space is local to the E2 node. In some embodiments, the RIC request ID number space is a temporary as well as a permanent component. For example, a RIC request ID used for display reports persists even if a RIC request ID used from a subscription is reused.

Table 1 below shows example message formats used in some embodiments for communication between the RIC and the RIC SDK. This is the type of encapsulation header used to encapsulate all messages sent to and received from the RIC SDK. In some embodiments, the encapsulation header stores data needed by the RIC SDK and RIC for efficient processing of data messages. In the example shown in Table 1, the first 16 bytes associated with msg_type, msg_serial_num, msg_len, msg_flags, ctrl_len are part of the encapsulation header along with the ctrl_info field. The payload of the encapsulated packet can contain arbitrary data. In the example shown in Table 1, payload includes the original E2AP packet and E2SM payload.

All messages between the RIC and the RIC SDK are encapsulated in the headers shown in Table 1. Control information and payload are optional. Some messages have control information but no payload field, some messages have a payload without control information, and some messages have both a control field and a payload field. In some embodiments, the RIC SDK can be configured to trap these messages and reformat them for presentation to xApps. The format of the message is a raw byte stream. In some embodiments, the message CRC field is not used, and in other embodiments it is used.

The near RT RIC 3200 handles connection, disconnection, reset, and crash of the E2 node and xApp as follows. For E2 nodes, the RIC of some embodiments handles connections, disconnections, and crashes similarly. Specifically, when a connection to an E2 node is dropped and comes back up for any of these reasons, the E2 node sends the connection configuration again as if it had originally started up, and the near RT RIC Clean up all state related to the node and start over. In some embodiments, the near RT RIC notifies all xApps when an E2 node connection drops and comes back, regardless of whether they were previously subscribed to a particular E2 node. This is because the E2 node may advertise new features that may be of interest to xApps that were not previously subscribed. When the xApp connects, disconnects, or crashes, the near RT RIC performs the same operation again. Reset all state of the xApp in the near RT RIC and remove its subscription from all E2 nodes.

FIG. 33 shows the E2AP message processing of the near RT RIC 3200 datapath pod. In the following discussion of E2AP message processing and other message processing, for the sake of brevity, this datapath pod will simply be referred to as the near RT RIC 3200 or RIC 3200. As shown, the near RT RIC 3200 first decodes the E2AP message received from the E2 node. In some embodiments, the decoding includes decoding only the E2AP header, while in other embodiments, the decoding includes decoding the E2AP header and the E2SM payload. For some or all of this decoding (eg, E2SM), the near RT RIC of some embodiments uses a hardware accelerator (eg, a GPU accelerator) that is accessed via the bypass path described above.

After decoding the E2AP message, the Near RT RIC creates or updates the internal data structure considering the received data message and then creates a flat encapsulated message to the xApp in the above format with reference to Table 1. create. Because the Near RT RIC and RIC SDK run on different containers, and in some embodiments reside on different pods, they do not pass any data structures to each other, and in some embodiments their data Format the exchange into an encapsulated message with a specific sequence of bytes. After encapsulating the data message, the near RT RIC forwards the data message to the xApp pod, whose RIC SDK forwards it to the appropriate xApp.

Internal data structures created or updated in the near RT RIC during processing of an E2AP message are used for processing response messages from the xApp to the E2AP message and for processing subsequent E2AP messages. In some embodiments, examples of data stored in the near RT RIC's internal data structures include (1) a subscription list of xApps that are interested in data from a particular E2 node; a specific data tuple of interest from the E2 node; (3) a record identifying the network address and other location data associated with the E2 node and the xApp; (4) a reserved and assigned identifier (e.g., RIC request ID). including.

When an xApp sends a message, its RIC SDK processes the message and forwards it to the RIC along the shared memory or SCTP interface as described above. The near RT RIC then parses the message and stores the parsed components. Based on these components and one or more data tuples stored in the internal data structure of the associated E2 node message, the RIC creates an E2AP response, encodes this response, and sends it to the destination E2 node. Transfer to.

For example, after the first xApp sends a subscription request to receive M data tuples from the E2 node, create the state of the near RT RIC's data path pod to receive the desired subscription for the first xApp. record and request a subscription using an E2 node for M data tuples, forward these M data tuples to the xApp the first time they are received, and then forward these M data tuples to the xApp each time they are received. Transfer M data tuples. In some embodiments, the datapath pod of the near RT RIC may be configured to forward M data tuples to the associated SDL whenever it receives from the E2 node.

After the first xApp subscribes to receive M data tuples from the E2 node, the second xApp subscribes to receive N different data tuples from the E2 node, where N is greater than M. can do. The near RT RIC sends the updated subscription request to the E2 node. With this update, N data tuples are now required. Every time the near RT RIC receives N data tuples, M data tuples are sent to the first xApp and all N data tuples are sent to the second xApp.

Another example is to delete the near RT RIC and cache the RIC request ID from the E2AP message from the E2 node in response to the subscription request. Once this ID is removed, the RIC provides part of the E2AP message and its E2SM payload (if applicable) to the xApp. Later, when the xApp deletes the subscription, the RIC retrieves the RIC request ID from its state and inserts it into the E2AP message to the E2 node to request deletion of the subscription.

In some embodiments, the near RT RIC's processing of E2 setup, response, and failure messages is as follows: The near RT RIC first receives a setup request message from the E2 node. In response, the near RT RIC decodes the message and builds internal data structures. The RIC also caches the raw ASN1 payload. In some embodiments, the near RT RIC accepts any added RAN function identifiers. In some embodiments, after decoding the E2AP header, the near RT RIC sends the setup message to the xApps without decoding anything else (i.e., as a message with an ASN1 encoded E2SM payload). In some embodiments, the setup message that the near RT RIC sends to the xApps has a control length (ctrl_len) of 0 along with its ASN1 encoded payload.

When the xApp later connects, the near RT RIC sends all setup requests from the E2 nodes to the xApp and maintains an inventory of connected E2 nodes. In some embodiments, the near RT RIC sends these messages one at a time. Also, as described above, the near RT RIC in some embodiments constructs an E2Setup response and sends it to the E2 node. In some embodiments, the near RT RIC sends a failure message when a setup request is malformed (e.g., removes a record that is a duplicate of the RAN function list or has not been added to the list). ).

After receiving the reset from the E2 node, the near RT RIC decodes the message and then performs the following operations. Send a message about this reset to all xApps that have subscriptions to this E2 node. In some embodiments, this is an internal message without ASN 1 content. The Near RT RIC finishes sending the Delete Subscription message to the E2 node for all previous subscriptions. It also sends control, insertion, and policy deletion to this E2 node. Clean up any outstanding requests and send a reset response to the E2 node.

The Near RT RIC also employs service update, confirmation, and failure messages in some embodiments. This message updates the supported RAN functions list with additions, modifications, and deletions. The near RT RIC informs all xApps about the new service configuration of the E2 node. In some embodiments, the near RT RIC sends a message to the xApps after applying the configuration, so it reflects the final state of the configuration. In other embodiments, the near RT RIC sends messages as is to the xApps to calculate the difference between the previous state and the new state of the supported RAN functions. In the latter approach, the near RT RIC does not need to ASN1 encode the resulting delta.

Processing of E2AP subscriptions is as follows in some embodiments. xApp formats the E2SM part of the subscription and ASN1 encodes it. Table 2 below details the control part of the subscription message (i.e., the part stored in the control field of the xApp to Near RT RIC message). The payload will be E2SM content encoded with ASN1. In some embodiments, multiple subscription message types are defined to clarify optional information. Also, in some embodiments, message flags are used to specify the exact format. In some embodiments, each join message specifies one E2 node global ID and one RAN capability ID.

In some embodiments, the E2 node sends a 113 byte identifier and the RIC compresses it into a 40 byte ID. When sending the subscription message to the E2 node, the RIC converts the 40 bytes into a 113 byte ID. Subscription message control fields will be in a fixed format wherever possible. In some embodiments, the RIC caches all subscription requests and compares requests from multiple xApps to avoid sending out duplicate subscription messages. However, if after the first initial xApp, a second subsequent xApp requests additional information from the same E2 node, the RIC will resend the subscription (using the same RIC request ID in some embodiments) . However, this time we will request additional information. When sending a subscription request to an E2 node, the RIC, in some embodiments, sends the entire payload received from the xApp as an E2AP message payload.

The near RT RIC saves the E2 node's global ID and the RIC request ID (generated by the RIC) as control information and sends an exactly ASN1 encoded message from the E2 node back to the xApps to respond to the E2AP RIC subscription response. process.

The E2AP RIC subscription deletion request, response, or failure message is sent from the xApp to the near RT RIC with message fields sent as control information (ie, as part of ctrl_info). The near RT RIC creates an encoded ASN1 message and sends it to the E2 node. The deletion request does not specify the E2 node global ID. Therefore, this information is provided by the xApp's RIC. In some embodiments, the response message is sent from the near RT RIC to the xApp as packed bytes (not ASN1 encoded).

The E2AP notification report of an E2 node is processed as follows: The message is decoded by the near RT RIC and the RIC Request ID field is determined. This helps to determine the xApp that has subscribed to the indication. The near RT RIC in some embodiments sends the message as ASN1 encoded into the xApp. In some embodiments, the near RT RIC also sends the reduced E2 Global ID as control information with the message.

Near RT RIC processing of E2AP control requests is as follows in some embodiments. xApp sends this request as packed byte information. In the Near RT RIC, the E2 Global ID is not specified in the message since this information is specified by the xApp. The near RT RIC format of this message is shown in Table 3.

The Near RT RIC processes E2AP control responses or failure messages as follows. The near RT RIC decodes the message and obtains the RIC request ID. Next, an ASN1 encoded message with the global E2 node ID added to the beginning as control information is sent to the xApp.

In some embodiments, the SDL data store is in a memory database running on a unique set of one or more pods. Has its own compute and memory resources allocated. As mentioned above, multiple near RT RIC instances define a distributed near RT RIC. In some embodiments, each near RT RIC instance has its own instance of SDL that stores system-wide information for the RIC instance. Examples of such information include a list of connected E2 nodes (ie, base station nodes), xApps, subscriptions from each xApp, and critical cell data returned by the E2 nodes. Additionally, each SDL instance in some embodiments internally executes custom algorithms as data arrives, by interfacing to hardware accelerators (e.g., GPUs), or by post-processing data retrieved from its storage. , provides services to preprocess incoming data.

The RIC instance's data IO pod and xApp pod are connected to the RIC instance's SDL pod. In some embodiments, each SDL instance operates only on its own RIC instance's data IO pods and service pods. Also, in some embodiments, SDL pods are managed by the SMO and configured via service pods. Data flow to and from SDL includes (1) data IO to SDL data storage, (2) xApps from SDL data storage, (3) xApps to SDL data storage, (4) data IO from SDL data access. (obtaining E2 node information, subscription information, etc.), and (5) providing and obtaining configuration information by the service pod with SDL communication.

FIG. 34 illustrates some embodiments of a RIC instance 3400 with an SDL pod 3410. As described, SDL 3410 includes a configuration agent 3412, an SDL data path agent 3414, a RIC SDK agent 3417, SDL pre- and post-processors 3416 and 3418, and one or more SDL data stores 3420. SDL configuration agent 3412 interfaces with service pod agent 3430. This agent 3412 configures and manages other components of the SDL pod. SDL pods are managed by SMO and configured via service agent 3430 in service pod #090.

SDL datapath agent 3414 is the datapath interface that the SDL pod exposes to the control thread and datapath thread of datapath pod 3450. SDL data path agent 3414 handles communications from these entities on behalf of SDL and performs reads and writes to SDL data store 3420 on behalf of these data path entities. In some embodiments, the SDL datapath agent may be configured to communicate with the datapath pod 3450 using either SCTP or a shared memory library.

In some embodiments, the RIC SDK agent 3417 is an agent that the SDL pod exposes to the RIC SDK of the xApp pod. The RIC SDK agent 3417 handles communication from the RIC SDK to the SDL and performs reads from and writes to the SDL data store 3420 for the RIC SDK. In some embodiments, the RIC SDK agent 3417 may be configured to communicate with the RIC SDK using either SCTP or a shared memory library. This agent 3417 also performs cache synchronization operations to synchronize the RIC SDK's SDL cache with the SDL data store 3420. Additionally, if xApps need to scale to dozens of connections, the connection manager of some embodiments may be configured to handle the RIC instance's epoll connections (e.g., the epoll used by the data IO pods of some embodiments). connection processing).

SDL agents 3414 and 3417 handle event subscriptions and notifications from the RIC SDK and datapath pods. This is separate from E2AP subscription management, but conceptually similar. For example, through this subscription service of SDL, xApps specify their interest in some data via a key or frequency of reports. The RIC SDK agent 3417 then provides periodic updates to this xApp based on the subscription. It also provides security services in some embodiments by encrypting and decrypting data.

Data pre-processors and post-processors 3416 and 3418 are part of SDL 3410 in some embodiments to flexibly execute certain value-added algorithms on the data. In some embodiments, both of these processors 3416 and 3418 operate in one container, and in other embodiments, each of them operates in separate containers within SDL pod 3410. Each of these processors also interfaces with an external accelerator (eg, GPU 3480) to perform their operations in some embodiments. Data preprocessor 3416 is executed inline when data is stored in SDL data store 3420.

In some embodiments, data post-processor 3418 is executed inline when data is read from SDL data store 3420. Alternatively, post-processor 3418 in some embodiments can be configured to operate in the background on data stored in SDL data storage 3420 (e.g., retrieve data from this data storage, perform some operation on the data and store the result in data storage). The data processor of some embodiments encodes and/or decodes the E2SM payload of the E2AP message. This is advantageous because the Datapath pod can pass the ASN1 string to the SDL for decoding and storage. As mentioned above, the RIC SDK in some embodiments may also be configured to provide E2SM encoding/decoding services.

Another example of a post-processor operation that data processor 3418 performs in some embodiments is a machine-trained operation. In some embodiments, the post-processor 3418 collects various data tuples stored by various xApps and/or various pods (e.g., data path pods) and applies these data tuples to a machine learning network (e.g., , a neural network trained by machine learning). In some embodiments, post-processor 3418 uses one or more hardware accelerators (e.g., one or more GPUs) of the SDL host computer to execute its machine training network operations. Execute. Post-processor 3418 accesses the hardware accelerator through the bypass approach described above with reference to FIGS. 14-20.

Post-processor 3418 may pass the results of its operations to one or more xApps or store the results in SDL data store 3420 for retrieval by one or more xApps. Examples of results obtained by post-processing SDL data with machine training networks include anomaly detection (e.g. identifying E2 nodes that are behaving abnormally, e.g. cell sites suddenly receiving too many connections) .

Using the machine training network within the SDL pod 3410 to process the different xApp outputs stored in the SDL data store 3420 means that this data store 3420 can process a large number of data provided by the E2 nodes based on the xApp subscription. Similar to tuples, it stores the output of several xApps, so it has advantages. In many cases, an individual xApp only knows the data tuples to which it subscribes and the results of its own operations and outputs of some other xApps. On the other hand, SDL pod 3410 has access to a larger set of E2 node input data and xApp output data. Instead of using machine-trained networks to perform such post-processing, post-processor 3418 uses algorithms (e.g., constrained optimization solvers) that are stored in SDL data storage 3420. Post-process the data. In other words, the post-processor 3418 of some embodiments does not use a machine-trained network and still uses its host computer's hardware accelerator (e.g., via a bypass path) to perform its operations. Execute.

Some embodiments also use the post-processor 3418 to provide the current state of the E2 node when the first xApp begins subscribing to the state of the E2 node. After previously subscribing the second xApp to receive the state of the E2 node, the near RT RIC stores multiple data tuples related to the state of this node over a period of time. When the first xApp subsequently subscribes to the state of the E2 node and either this xApp or the datapath pod attempts to access this state of the first xApp, the post processor 3418 Obtain all previously stored data tuples for and use these data tuples to compute the current state of the E2 node. This then provides the first xApp either directly or via a datapath pod.

SDL data store 3420 is an in-memory database. In some embodiments, the memory in the memory database is a database that is loaded into and run from computer system memory (eg, host computer non-volatile memory, including its RAM). One example of such a database is Redis. In some embodiments, the size of data storage is selected to minimize search and storage latency. When an existing data storage reaches its desired maximum size, some embodiments create additional instances of this data storage within the same or different instances of the SDL.

Also, as shown in FIG. 34, the SDL 3410 of some embodiments has active data storage 3420 and standby data storage 3421 for HA reasons. Additionally, some embodiments provide data storage 3420 that allows different SDL instances of different RIC instances to synchronize some or all of their data in the background for HA and/or data availability reasons. Make it. In some embodiments, xApps and RIC components read and write data to active SDL storage that is synchronized with standby SDL storage in the background. In the event of a failure of the active SDL storage, the RIC of these embodiments can seamlessly switch to standby SDL storage. Also, the RIC can switch to standby SDL storage when active SDL storage is upgraded. This allows SDL storage to be hitless.

As described above with reference to FIG. 13, the RIC SDK includes an SDL cache that provides local SDL storage for xApps and synchronizes its data with the SDL data store. In some embodiments, this RIC SDK cache pulls data in bulk and periodically from the main SDL data store. This can reduce the number of requests made to the main SDL pod. The RIC SDK cache also reduces the delay for reading SDL data by providing some of this data locally. The RIC SDK cache also reduces the time it takes to write data to the SDL by allowing data to be written locally in xApp pods and synchronized in the background. The size of the SDK cache in some embodiments is smaller than the SDL data storage 3420. Also, in some embodiments, this size is based on the requirements of the set of one or more xApps running on the pod with the RIC SDK.

In some embodiments, the sources of data to the SDL 3410 are accessed through (1) the control thread and datapath thread of the datapath pod 3450, (2) the xApp via the RIC SDK, and (3) the A1 interface. (4) xApp pod configuration storage, and (5) SMO configuration server. In some embodiments, examples of system data in the SDL (e.g., written by the control thread) are: (1) E2 node information, (2) cell information, (3) UE information, (4) E2 node report. , (5) KPM (Key Performance Metrics) reports, and (6) topology information (from the EMS adapter).

Examples of SDL transactions in some embodiments include (1) a data IO pod (control or data IO thread) that writes data to the SDL that is read by an xApp; (2) data from another xApp that is written to the SDL. an xApp that reads an an xApp that writes to the SDL to perform an operation; an xApp that retrieves the results of this operation from the SDL (e.g., by using GPU services or simply using a generic CPU); (4) one of the A1 subscriptions; (5) an SMO that stores 01 configuration data in the SDL; and (6) a non-RT RIC that stores ML data in the SDL.

FIG. 35 conceptually illustrates an electronic system in which some embodiments of the invention are implemented. Electronic system 3500 may be a computer (eg, a desktop computer, a personal computer, a tablet computer, a server computer, a mainframe, a blade computer, etc.) or any other type of electronic device. Such electronic systems 3500 include various types of computer readable media and interfaces to various other types of computer readable media. Electronic system 3500 includes a bus 3505, a processing unit 3510, a system memory 3525, a read only memory 3530, a persistent storage device 3535, an input device 3540, and an output device 3545.

Bus 3505 collectively represents all system, peripheral, and chipset buses that communicatively connect numerous internal devices of electronic system 3500. For example, bus 3505 communicatively connects processing unit 3510 with read-only memory 3530, system memory 3525, and persistent storage device 3535.

From these various memory units, processing unit 3510 retrieves instructions to execute and data to process in order to perform the processes of the present invention. Processing unit 3510 may be a single processor or a multi-core processor in various embodiments.

The read-only memory 3530 stores static data and instructions required by the processing unit 3510 and other modules of the electronic system 3500. On the other hand, the persistent storage device 3535 is a readable and writable memory device. This device 3535 is a non-volatile memory unit that stores instructions and data even when the electronic system 3500 is turned off. Some embodiments of the invention use mass storage devices (such as magnetic or optical disks and their corresponding disk drives) as the persistent storage device 3535.

Other embodiments use a removable storage device (floppy disk, flash drive, etc.) as the persistent storage device 3535. Like persistent storage device 3535, system memory 3525 is a read/write memory device. However, unlike storage device 3535, system memory 3525 is volatile read-write memory, such as random access memory. System memory 3525 stores some of the instructions and data that the processor needs during execution. In some embodiments, the processes of the present invention are stored in system memory 3525, persistent storage device 3535, and/or read-only memory 3530. From these various memory units, processing unit 3510 retrieves instructions to execute and data to process in order to perform the processing of some embodiments.

Bus 3505 also connects to input/output devices 3540 and 3545. Input device 3540 allows a user to communicate information and select commands to electronic system 3500. Input devices 3540 include alphanumeric keyboards and pointing devices (also referred to as "cursor control devices"). Output device 3545 displays images generated by electronic system 3500. Output devices 3545 include printers and display devices such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs). Some embodiments include devices such as touch screens that function as both input and output devices.

Finally, as shown in FIG. 35, bus 3505 also couples electronic system 3500 to network 3565 via a network adapter (not shown). In this way, the computer may be part of a network of computers, such as a local area network ("LAN"), a wide area network ("WAN"), an intranet, or the Internet. Electronic Any or all components of system 3500 can be used with the present invention.

Some embodiments include an electronic component, e.g., a microprocessor, a computer readable or computer readable medium (alternatively referred to as a computer readable storage medium, a machine readable medium, or a machine readable storage medium) storing computer program instructions. including storage and memory. Some examples of such computer readable media are RAM, ROM, compact disc read only (CD-ROM), compact disc recordable (CD-R), compact disc readable (CD-RW), read only digital General-purpose discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD) cards, micro-SD cards, etc.) and/or solid-state hard drives, read-only and recordable Blu-Ray disks, ultra-density optical disks, other optical or magnetic media, and floppy disks. A computer-readable medium can store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include files that include machine code, such as produced by a compiler, and higher-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

Although the above description primarily refers to a microprocessor or multi-core processor that executes software, some embodiments include one or more processors, such as an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). Executed by integrated circuits. In some embodiments, such integrated circuits execute instructions stored on the circuit itself.

As used herein, the terms "computer," "server," "processor," and "memory" all refer to electronic or other technological devices. These terms exclude a person or group of people. As used herein, the term display or representation refers to display on an electronic device. As used herein, the terms "computer-readable medium(s)," "computer-readable medium," and "machine-readable medium(s)" refer to any tangible medium that stores information in a computer-readable form. limited to physical objects as a whole. These terms do not include any wireless signals, wire-downloaded signals, or any other ad hoc signals.

Although the invention has been described with reference to numerous specific details, those skilled in the art will recognize that the invention can be embodied in other specific forms without departing from the inventive concept. Will. For example, some diagrams conceptually illustrate processes. Certain operations of these processes may not be performed in the exact order shown and described. Certain operations may not be performed in one continuous series of operations, and different specific operations may be performed in various embodiments. Furthermore, a process can be implemented using several subprocesses or as part of a larger macro process.

Also, some embodiments described above show only one hardware accelerator per host computer. However, one of ordinary skill in the art will appreciate that the methodology and architecture of some embodiments can be used to provide direct pass-through access to multiple hardware accelerators on a single host computer. You will recognize it. Additionally, some of the embodiments described above relate to xApp operations and near RT RIC communication with xApps. One of ordinary skill in the art will recognize that these embodiments are equally applicable to edge applications in telecommunications networks and near RT RIC communications with edge applications. Accordingly, those skilled in the art will appreciate that the invention is not limited by the above exemplary details, but rather is defined by the appended claims.

Claims (35)

A method of performing control plane operations in a radio access network (RAN), the method comprising:
Deploying multiple machines on a host computer and
Deploying a control plane application that performs control plane operations on each machine;
configuring a RAN intelligent controller (RIC) SDK that acts as an interface between the control plane application and a set of one or more elements of the RAN on the same machine. 2. The method of claim 1, wherein on each machine, the RIC SDK includes a set of network connection processes that establish a network connection to the set of RAN elements for the control plane application, A method for enabling said control plane application above to refrain from having said set of network connectivity processes. The method of claim 2, wherein the set of network connectivity processes of each RIC SDK of each machine establishes and maintains network connectivity between the machine and the set of RAN elements used by the control plane application of the machine, and controls data packet transmissions between the set of RAN elements for the control plane application. 2. The method of claim 1, wherein the control plane application on each machine accesses the set of RAN elements via high-level API calls that the RAN SDK translates into low-level API (application program interface) calls. A way to communicate with. 5. The method of claim 4, wherein at least a subset of the low-level API calls are defined by a standards organization. 5. The method of claim 4, wherein the high-level API calls are written in a high-level programming language, while the low-level API calls establish and maintain network connections and transfer data packets over these connections. A method, including low-level calls that pass through. 5. The method of claim 4, wherein the set of RAN elements includes CUs (centralized units) and DUs (distributed units) of the RAN. 8. The method of claim 7, wherein the RAN SDK on each machine communicates with the CU and the DU via the E2 interface defined in the low-level standardization, while the RAN SDK on the machine A control plane application communicates with the CU and the DU via the RAN SDK using high-level API calls at a high-level application layer that do not involve low-level transport or network operations. A method for defining E2 interface operation. 5. The method of claim 4, wherein the set of RAN elements includes network elements of the RIC. 10. The method of claim 9, wherein the RIC element includes at least one shared data layer (SDL) element. 10. The method of claim 9, wherein the RIC element includes at least one datapath input/output (I/O) element. 10. The method of claim 9, wherein the RIC element includes at least one service management element. A method for a control plane application to communicate in a RAN (Radio Access Network), the method comprising:
deploying multiple control plane applications running on multiple host computers;
deploying a plurality of RAN intelligent controllers (RICs) running on the plurality of host computers to implement a distributed RIC that acts as a communication interface between the control plane applications. 14. The method of claim 13, further comprising: receiving an application programming interface (API) call from at least a first control plane application and forwarding the API call to at least a second control plane application. 1. A method comprising configuring a RIC of 1. 15. The method of claim 14, wherein the first control plane application and the second control plane application run on the same host computer. 15. The method of claim 14, wherein the first RIC and the first control plane application run on a first host computer, and the second control plane application runs on a second host computer. configuring the first RIC from the first control plane application on the second computer such that the second RIC forwards to the second control plane application. A method comprising: configuring the first RIC to forward the API call to the second RIC for operation. 15. The method of claim 14, wherein the first control plane application and the second control plane application are two different parties that use a common set of RIC APIs to communicate with each other via the distributed RIC. Developed by application developers. 15. The method of claim 14, wherein configuring the first RIC comprises: when the first RIC forwards the API call from the first control application to the second control application. , configuring the first RIC to add one or more parameters to the API call. 15. The method of claim 14, wherein the first control plane application and the second control plane application run on a first machine and a second machine running on one host computer, The method further includes configuring on each machine a RIC SDK for receiving and forwarding API calls between the distributed RIC and the first control plane application and the second control plane application. . 20. The method of claim 19, wherein on each machine, the RIC SDK includes a set of network connection processes that establish a network connection to the set of RAN elements for the control plane application, A method for enabling said control plane application above to refrain from having said set of network connectivity processes. 21. The method of claim 20, wherein the set of network connection processes for each RIC SDK of each machine provides a link between the machine and the set of RAN elements used by the control plane application of the machine. A method for establishing and maintaining network connectivity and controlling data packet transmission to and from the set of RAN elements for the control plane application. 20. The method of claim 19, wherein the control plane application on each machine accesses the set of RAN elements via high-level API calls that the RAN SDK translates into low-level API (application program interface) calls. and wherein at least a subset of the low-level API calls are defined by a standards-setting body. 20. The method of claim 19, wherein the control plane application on each machine accesses the set of RAN elements via high-level API calls that the RAN SDK translates into low-level API (application program interface) calls. , and the high-level API calls are written in a high-level programming language, while the low-level API calls make low-level calls that establish and maintain network connections and pass data packets over these connections. Including, methods. A method for a control plane application to use shared data layer (SDL) storage in a radio access network (RAN), the method comprising:
deploying the control plane application running on a host computer;
deploying an SDL cache on the host computer;
using the SDL cache to process at least a subset of SDL storage access requests of the control plane application. 25. The method of claim 24, wherein the control plane application runs on a machine running on the host computer, and the SDL cache runs on the machine. 25. The method of claim 24, wherein the control plane application runs on a machine running on the host computer, and the SDL cache runs on the host computer. 25. The method of claim 24, deploying a RAN intelligent controller (RIC) operating on a host computer to synchronize data stored in the SDL cache with data stored in the SDL storage. A method further comprising: 28. The method of claim 27, wherein deploying the RIC includes deploying the RIC implementing a distributed RIC with other RICs running on other host computers. 29. The method of claim 28, wherein the SDL storage runs on a different host computer than the host computer on which the control plane application runs. 29. The method of claim 28, wherein at least a portion of the SDL storage runs on the host computer on which the control plane application runs. 25. The method of claim 24, wherein the control plane application runs on a machine running on the host computer, and the method further comprises processing storage access requests from the control plane application. , configuring a RIC SDK on the machine. 32. The method of claim 31, wherein the SDL cache is part of the RIC SDK. 33. The method according to claim 32,
Deploying the RIC includes deploying the RIC implementing a distributed RIC with other RICs running on other host computers;
The method, wherein the SDL storage is part of the distributed RIC. 34. The method of claim 33, wherein the RIC SDK forwards the SDL access request from the control plane application to the SDL cache if the RIC SDK is unable to process the SDL access request via the SDL cache. how to. 35. The method of claim 34, wherein if the SDL cache does not store data requested by the control plane application, the RIC SDK processes an SDL access request via the SDL cache. There is no way.

Images