WO2025201632A1

WO2025201632A1 - Testing for intent management system

Info

Publication number: WO2025201632A1
Application number: PCT/EP2024/058127
Authority: WO
Inventors: Pedro Henrique GOMES DA SILVA; Andrey José TORRES DA SILVA; Jörg NIEMÖLLER; Mirko D'ANGELO; Danesh DAROUI
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2024-03-26
Filing date: 2024-03-26
Publication date: 2025-10-02
Anticipated expiration: 2026-09-26

Abstract

Methods and devices for testing intents are disclosed including receiving, at an intent tester, a request from an intent owner to implement an intent test, the request including service requirements and test environment information. A set of intent systems under test are determined using the service requirements and the test environment information. An intent under test is generated for each of the intent systems using the service requirements and the test environment information. The intents under test are sent to the intent systems under test. Intent test reports are received from the intent systems under test including information about the intent systems testing the intents under test. A response is generated to the request to implement the intent test using the intent test reports from the intent systems under test. The response is sent to the intent owner.

Description

TESTING FOR INTENT MANAGEMENT SYSTEM

TECHNICAL FIELD

[0001] Embodiments of the invention relate to the field of autonomous networks; and more specifically, to intent management in autonomous networks.

BACKGROUND ART

[0002] Autonomous networks are networks and software platforms that can sense their environment and adapt their behavior accordingly with little to no human input. Conventional autonomous networks operate using an intent framework where intents are communicated between an intent owner and an intent handler within the autonomous networks. Intent reports are sent by intent handlers to the intent owners at major events in the intent lifecycle and intent fulfilment process and include the current status of the intent fulfilment. The information within the intent reports includes, for example, current measurement of key performance indicators and information relating to requirements requested by the intent owner.

SUMMARY

[0003] A method for an intent tester implemented by a first electronic device is disclosed. The method includes receiving, at the intent tester implemented by the first electronic device, a request from an intent owner to implement an intent test, wherein the request comprises service requirements and test environment information, determining a set of intent systems under test using the service requirements, generating an intent under test for each of the set of intent systems using the service requirements and the test environment information, sending, to the set of intent systems under test, the intents under test, receiving, from the set of intent systems under test, intent test reports in response to sending the intents under test, wherein the intent test reports include information about the set of intent systems testing the intents under test, generating a response to the request to implement the intent test using the intent test reports from the intent systems under test, and sending, to the intent owner, the response.

[0004] A first electronic device implementing an intent tester system is disclosed, the first electronic device including a processor and a memory, the memory containing instructions executable by the processor whereby the first electronic device is operative to receive, at the intent tester implemented by the first electronic device, a request from an intent owner to implement an intent test, wherein the request comprises service requirements and test environment information, determine a set of intent systems under test using the service requirements, generate an intent under test for each of the set of intent systems using the service requirements and the test environment information, send, to the set of intent systems under test, the intents under test, receive, from the set of intent systems under test, intent test reports in response to sending the intents under test, wherein the intent test reports include information about the set of intent systems testing the intents under test, generate a response to the request to implement the intent test using the intent test reports from the intent systems under test, and send, to the intent owner, the response.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings:

[0006] Figure 1 illustrates an exemplary intent testing environment, according to some embodiments of the invention.

[0007] Figure 2 illustrates an exemplary intent testing flow with an intent tester system and an intent system under test, according to some embodiments of the invention.

[0008] Figure 3 illustrates another exemplary intent testing flow with an intent tester system and an intent system under test, according to some embodiments of the invention.

[0009]

[0010] Figure 4 illustrates an exemplary intent tester system, according to some embodiments of the invention.

[0011] Figure 5 illustrates another exemplary intent testing environment, according to some embodiments of the invention.

[0012] Figure 6 is a flow diagram of an example method to test intents in a test management system, according to some embodiments of the invention. [0013] Figure 7A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments of the invention.

[0014] Figure 7B illustrates an exemplary way to implement a special-purpose network device according to some embodiments of the invention.

[0015] Figure 7C illustrates various exemplary ways in which virtual network elements (VNEs) may be coupled according to some embodiments of the invention.

[0016] Figure 7D illustrates a network with a single network element (NE) on each of the NDs, and within this straightforward approach contrasts a traditional distributed approach (commonly used by traditional routers) with a centralized approach for maintaining reachability and forwarding information (also called network control), according to some embodiments of the invention.

[0017] Figure 7E illustrates the simple case of where each of the NDs implements a single NE, but a centralized control plane has abstracted multiple of the NEs in different NDs into (to represent) a single NE in one of the virtual network(s), according to some embodiments of the invention.

[0018] Figure 7F illustrates a case where multiple VNEs are implemented on different NDs and are coupled to each other, and where a centralized control plane has abstracted these multiple VNEs such that they appear as a single VNE within one of the virtual networks, according to some embodiments of the invention.

[0019] Figure 8 illustrates a general-purpose control plane device with centralized control plane (CCP) software 850), according to some embodiments of the invention.

[0020] Figure 9 illustrates an apparatus including a processor, according to some embodiments of the invention.

[0021] Figure 10 illustrates an exemplary Open Radio Access Network (ORAN) compliant network, according to some embodiments of the invention.

DETAILED DESCRIPTION

[0022] The following description describes methods and apparatus for testing intents in an intent management system. In some embodiments, intent tests for an intent management system are described. In the following description, numerous specific details such as logic implementations, opcodes, means to specify operands, resource partitioning/sharing/duplication implementations, types and interrelationships of system components, and logic partitioning/integration choices are set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.

[0023] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0024] Bracketed text and blocks with dashed borders (e.g., large dashes, small dashes, dotdash, and dots) may be used herein to illustrate optional operations that add additional features to embodiments of the invention. However, such notation should not be taken to mean that these are the only options or optional operations, and/or that blocks with solid borders are not optional in certain embodiments of the invention.

[0025] In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other.

[0026] Intent management systems are autonomous network systems including multiple intent managers that can make decisions and take appropriate actions based on goals, requirements, and constraints of the intent management system and its constituent intent managers. These intent management systems can operate on multiple levels (e.g., hierarchically) such that a first intent manager may set goals, requirements, and constraints for a second intent manager that in turn sets goals, requirements, and constraints for a third intent manager. These intent managers communicate using intents, which include the requirements, goals, and constraints for a pair of intent managers. The intent manager that sends the intent is referred to as the intent owner whereas the intent manager receiving the intent is referred to as the intent handler. Intent management systems include interfaces that allow the intent managers to exchange, negotiate, and manage these intents. The intent owner sets the requirements, goals, and constraints for the network operation and the intent handler fulfills these requirements and goals in light of the constraints and sends intent reports back to the intent owner detailing the fulfillment of the intent. Even though the expectations (e.g., requirements, goals, and constraints) are set by the intent owner, the actions and processes that are performed based on those expectations are determined by the intent handler.

[0027] Embodiments of the invention support intent tests for intent management systems. The inclusion of intent testing in intent management systems is advantageous over conventional systems that do not support or include this information. Conventional intent management systems without intent testing capabilities are not able to test an intent system for hypothetical, past, or future circumstances and instead can only get information for a current state of the intent system in operation. These conventional systems perform sub optimally when future network conditions do not align with the current network conditions, even if the change in network conditions is foreseeable. For example, large events require significantly more network resources than otherwise required during normal network operation. Conventional intent management systems without the ability to test performance under different network conditions cannot properly allocate resources for these events, resulting in the network being overwhelmed (e.g., not delivering packets fast enough to meet requirements and/or dropping packets). Intent owners in conventional intent management systems can use probe intent requests to determine how intent handlers will react to a given set of requirements. For example, intent owners can send intent handlers probe intent requests as described in “IG1253C Intent Life Cycle Management and Interface vl .1.0”, Autonomous Networks Project, TM Forum, November 2021. The intent handler, however, evaluates the probe intent request and sends a response intent report based on the current network conditions (e.g., number of network devices, load of network devices, etc.). This inability to test hypothetical or future network conditions prevents these conventional intent management systems from optimally allocating resources to satisfy future requirements and/or network conditions. This restricts the ability of the intent management systems to adapt to future changes and creates an input lag when faced with changing network conditions and/or business rules. This also decreases trust in the intent management systems due to a general lack of knowledge of how the system will react in certain environments. This lack of trust can prevent the widespread use of intent management systems. Because these systems operate using machine learning principles, the less use these systems receive, the less effective and accurate they are. Accordingly, the lack of trust results in less accurate systems, furthering the lack of trust and adding to the feedback loop. Additionally, by allocating based on current resources, rather than future resources, intent management systems that do not use intent tests need to reallocate resources more often, resulting in inefficiencies such as taking up CPU cycles and network bandwidth.

[0028] In contrast, intent management systems that use intent testing are able to simulate different network conditions and business rules and proactively allocate resources within the intent management system to prevent potential reductions in the quality of service. These intent management systems can provide intent reports for hypothetical intents that allow operators to determine how the intent management system will react to potential environments and therefore increase the operator’s trust in these systems. Additionally, intent management systems using intent tests can allocate resources based on future needs, reducing the inefficiencies associated with reallocating resources due to changing network conditions. Intent testing, therefore, improves the operation of the intent management system and associated network by preventing networks from being overwhelmed and increasing the trust of these intent management systems, resulting in more use and therefore more accurate systems.

[0029] Figure 1 illustrates an exemplary intent testing environment, according to some embodiments of the invention. As shown in Figure 1, intent testing environment 100 includes intent owner 105, service requirements storage 115, intent tester system 125, intent system under test 135, network 145, and virtual network 155. In some embodiments, intent owner 105 is an operator of intent tester system 125. For example, intent owner 105 specifies test environment information including business rules and other parameters, together with service requirements for different types of services to be tested by intent tester system 125. These services are services to be tested 135 in a network such as network 145 or virtual network 155.

[0030] In some embodiments, test environment information 106 and/or service requirements 104 specified by intent owner 105 are stated at a high level and include technical and nontechnical information. For example, test environment information 106 and/or service requirements 104 can be expressed as an intent object, a set of objects formulates using XML/JSON, or even as natural language sentences. In some embodiments, intent owner 105 specifies services 102 to be provided by the intent system under test 135. These services 102 can include, as examples, a gaming service, an enhanced mobile broadband service (eMBB), and a mobile internet of things (mloT) service for a smart home. In some embodiments, intent owner 105 retrieves service requirements 104 from service requirements storage 115 based on services 102. For example, service requirements storage 115 is a storage device including service requirements for different services. In one example, service requirements storage 115 includes a requirement for latency and a requirement for a number of user equipment (UE) served. In such an embodiment, intent owner 105 retrieves service requirements 104 from service requirements storage 115 for services 102 including gaming, eMBB, and mloT. For example, intent owner 105 retrieves a latency and UE requirement for each of gaming, eMBB, and mloT services 102. In some embodiments, intent owner 105 sends services 102 to intent tester system 125 and intent tester system 125 retrieves service requirements 104 from service requirements storage 115 using the received services 102.

[0031] In some embodiments, intent owner 105 sends test environment information 106 to intent tester system 125. For example, test environment information 106 includes business rules such as legal requirements (e.g., relevant regulatory rules), service level agreements, and other contract requirements. In some embodiments, test environment information 106 includes the scope of the intent test. For example, test environment information 106 specifies what aspect of intent system under test is being tested (e.g., performance of the intent system, security of the intent system, privacy of the intent system, etc.). In some embodiments, test environment information 106 includes other aspects of the testing environment. For example, test environment information 106 specifies that the test should be implemented with a traffic load higher than 50% of load capabilities for the system.

[0032] Intent tester system 125 receives test environment information 106 and service requirements 104 and generates intent under test 108. For example, intent tester system 125 generates an intent object that includes test requirements (e.g., test requirements 315 of Figure 3) and test conditions (e.g., test conditions 320 of Figure 3). In some embodiments, intent tester system 125 generates test conditions using test environment information 106 received from intent owner 105. For example, intent tester system 125 translates the business rules and scope of test received from intent owner 105 into a specific network scenario that can be tested by intent system under test 135. In one example, intent tester system 125 translates a requirement by test environment information 106 to implement the test with a traffic load higher than 50% of load capabilities into test conditions including testing the system during business hours and/or a major sporting event. In some embodiments, intent tester system 125 is implemented by an operations layer of an autonomous network (e.g., business operations 510, service operations 515, or resource operations 525 of Figure 5). In such embodiments, intent tester system 125 translates test environment information 106 received from intent owner 105 into a specific network scenario (e.g., test conditions 320 of Figure 3) using a local intelligence component (e.g., local intelligence component 513, local intelligence component 518, domain intelligence components 528, or domain intelligence components 533 of Figure 5). For example, intent tester system 125 uses reasoning procedures such as reasoning inference and/or machine reasoning to translate test environment information 106 into test conditions. In some embodiments, as explained further with reference to Figure 4, intent tester system 125 translates test environment information 106 into test conditions using trained machine learning models (e.g., machine learning models 425 of Figure 4).

[0033] In some embodiments, intent tester system 125 generates the test requirements using test environment information 106 and service requirements 104. For example, intent tester system 125 receives test environment information 106 including requirements for compliance with General Data Protection Regulation (GDPR) of the European Union and receives service requirements 104 including a minimum throughput requirement of 1 Megabit per second (Mbps) for mloT service. Intent tester system 125 generates test requirements including a data privacy compliance requirement and a minimum throughput requirement. In some embodiments, intent tester system 125 generates test requirements including best value requests. For example, intent tester system 125 generates test requirements including specific minimum requirements for data privacy compliance and throughput with a best value request for minimizing energy consumption and maximizing geographical coverage. In such an embodiment, intent system under test 135 responds to intent under test 108 with intent test report 110 including the best values for minimizing energy consumption and maximizing geographical coverage given the test requirements and test conditions.

[0034] In some embodiments, intent tester system 125 generates reporting requirements for intent under test 108. For example, intent under test 108 includes reporting requirements including the reporting rate, periodicity, and other reporting characteristics for when and how intent system under test 135 should send intent test report 110 in response to intent under test 108. In some embodiments, intent under test 108 also includes a termination condition. For example, intent under test 108 includes a termination condition to terminate the intent test after two weeks. In such embodiments, intent system under test 135 determines to terminate intent under test 108 when the termination condition is met. For example, intent system under test 135 determines to terminate intent under test 108 in response to determining that two weeks have passed.

[0035] Intent tester system 125 sends the generated intent under test 108 (including test requirements and test conditions) to intent system under test 135. Intent under test 108 functions as an intent object with intent tester system 125 acting as the intent owner for the intent and intent system under test 135 acting as the intent handler. In some embodiments, as explained above, intent under test 108 includes test conditions and test requirements. For example, test conditions are the conditions in which the tests should be executed (e.g., network traffic load, time frame etc.) and test requirements include requirements to be considered during the test (e.g., number of UEs, geographical coverage, data throughput, energy consumption, data privacy compliance, etc.).

[0036] In some embodiments, intent tester system 125 modifies intent under test 108. For example, after sending intent under test 108 to intent system under test 135, intent tester system 125 can modify aspects of intent under test 108 (e.g., test conditions and/or test requirements). [0037] Intent system under test 135 receives intent under test 108 and performs an intent test on network 145 and/or virtual network 155 using the specified test conditions and test requirements. In some embodiments, intent system under test 135 performs the test on a real network 145 managed by intent system under test 135. For example, intent system under test 135 including an intent management component (e.g., intent management component 526 of Figure 5) managing multiple network elements (e.g., network element 540 of Figure 5) comprising network 145. In such an embodiment, intent system under test 135 can wait until the conditions of network 145 match the test conditions specified by intent under test 108 (e.g., intent system under test 135 has the autonomy to determine when to perform the intent test). For example, intent system under test 135 can wait until business hours and/or a major sporting event to implement the test on network 145. In other embodiments, intent system under test 135 performs the test on virtual network 155 which functions as a digital twin of network 145 managed by intent system under test 135. In such an embodiment, intent system under test 135 generates virtual network 155 which function as a digital twin of network 145 operating under the test conditions specified by intent under test 108. For example, intent system under test 135 generates virtual network 155 using data from prior operation of network 145. In such embodiments, intent system under test 135 then performs the intent test on virtual network 155 to predict how network 145 would function under the test conditions specified by intent under test 108. In some embodiments, intent system under test 135 determines to postpone the intent test or preempt the intent test. For example, if network conditions are not favorable (e.g., there is a high network load), intent system under test 135 can postpone/preempt the intent test to prevent any service interruptions and/or a reduction in quality of service for intent system under test 135.

[0038] In response to testing intent under test 108 on network 145 and/or virtual network 155, intent system under test 135 generates intent test report 110 including test results (e.g., test results 325 of Figure 3) and network conditions (e.g., network conditions 330 of Figure 3). In some embodiments, in response to postponing/preempting the intent test, intent system under test 135 sends intent test report 110 to intent tester system 125 indicating that the intent test has been postponed/preempted. The test results generated by intent system under test 135 are not necessarily a real service that can be consumed but rather a hypothetical outcome of what would happen if such an intent were received by intent system under test 135. For example, intent under test 108 can include requirements for an eMBB service with a minimum bandwidth requirement of 1 Gigabit per second (Gbps) and the test results of intent test report 110 indicate a maximum available bandwidth of 450 Mbps. In such an example, the eMBB service is not necessarily currently available at that bandwidth, but rather the intent test report 110 indicates the bandwidth that would be feasible to be provided by intent system under test 135 under the conditions specified by intent under test 108 should the eMBB service be requested (e.g., through an intent request).

[0039] In some embodiments, intent system under test 135 generates intent test report 110 including test results from intent under test 108. In one example, the test results indicate a range of numbers for UEs in the network (e.g., network 145 and/or virtual network 155) during the test, a geographical coverage area, a data throughput rate (or an indication of whether the specified data throughput rate was met), an energy consumption value, and an indication of whether data privacy requirements were met. In some embodiments, intent system under test 135 generates intent test report 110 including the network conditions for the network (e.g., network 145 and/or virtual network 155) during intent testing. For example, intent test report 110 includes network conditions indicating that the intent was tested over the period of three weeks and that there were two major events during the intent test period. In some embodiments, intent system under test 135 determines intent test report 110 (including test results and network conditions) using test results from downstream intent management components. For example, intent system under test 135 is implemented at a service operations layer of an autonomous network (e.g., service operations 515 of Figure 5). In such an embodiment, intent system under test 135 sends its own intent under tests to multiple intent management components of a resource operations layer of the autonomous network (e.g., intent management component 526 and intent management component 531 of resource operations 525 of Figure 5) and receives intent test reports from each of the intent management components. The intent test reports sent by the intent management components of the resource operations layer include details on the test results and the network conditions for each of the networks managed by the resource operations layer (e.g., network elements 540 and 545 of Figure 5). In such an embodiment, intent system under test 135 receives the intent test reports and generates intent test report 110 using the test results and network conditions for both of the intent test reports. Further details with regard to generating intent test report using downstream intent test reports are described with reference to Figure 2. [0040] Figure 2 illustrates an exemplary intent communication flow with intent owner 105, intent tester system 125, and intent system under test 135, according to some embodiments of the invention. As shown in Figure 2, intent communication flow includes intent owner 105, intent tester system 125, intent system under test 135, and target 220. Intent owner 105 is an entity that sends start intent test request 225. In some embodiments, intent owner 105 is the topmost intent owner of an autonomous network. In other embodiments, intent owner 105 is an intent management component, such as intent management component 511, 516, or 526 of Figure 5. In some embodiments each of intent owner 105, intent tester system 125, and intent system under test 135 are implemented in different operational layers or autonomous domains of a system. For example, intent owner 105 belongs to business operations, such as business operations 510 of Figure 5, intent tester system 125 belongs to service operations, such as service operations 515 of Figure 5, and intent system under test 135 belongs to resource operations, such as resource operations 525 of Figure 5. As an alternative example, intent owner 105 corresponds with intent owner 505 of Figure 5, intent tester system 125 with business operations 510 of Figure 5, and intent system under test with service operations 515 of Figure 5.

[0041] Target 220 is a managed entity of intent system under test 135. For example, if intent system under test 135 is implemented as an intent management component of service operations (e.g., intent management component 516), target 220 is an intent management component of a managed autonomous domain (e.g., intent management component 526). As an alternative example, if intent system under test 135 is implemented as an intent management component of an autonomous domain (e.g., intent management component 526), target 220 is a network element of that autonomous domain (e.g., network element 540 and/or a network element of network 145 and/or virtual network 155). In some embodiments, target 220 includes a machine learning model 222 trained to generate test results (e.g., test results 325) in response to execute test 250. In such embodiments, test results 255, intent test report 110, and intent test report 265 include machine learning model predictions or outputs based on how a network element modeled by target 220 would behave and/or did behave in response to execute test 250. In some embodiments, target 220 uses site intelligence (e.g., site intelligence 541 or 546 of Figure 5) to execute test and generate test results 255. For example, site intelligence 541 includes a trained machine learning model (e.g., model 222) which uses requirements from the intent test as inputs (e.g., test requirements of intent under test) and executes the test using the test requirements.

[0042] Intent owner 105 is the entity initiating the intent testing by intent tester system 125 and intent tester system 125 is the entity implementing the intent test on intent system under test 135. In the context of intent owners and intent handlers, intent owner 105 is implemented as the intent owner for start intent test request 225 and intent tester system 125 is implemented as the intent handler for start intent test request 225. As explained above, the intent owner of one intent may be an intent handler of a different intent. Similarly, the intent handler of one intent may be an intent owner of a different intent. Accordingly, although intent tester system 125 is implemented as the intent handler for start intent test request 225, intent tester system 125 is implemented as the intent owner and intent system under test 135 as intent handler 216 for intent under test 108. Although only limited layers are shown (e.g., intent owner 105, intent tester system 125, intent system under test 135, and target 220), actual implementations may have more or fewer layers depending on the requirements set forth in the intent test and the system architecture for the autonomous network.

[0043] In some embodiments, the external intent API includes functions organized into intent setting, intent negotiation, intent reporting, and profile handling. Intent setting functions include functions for creating, modifying, or deleting intents and for retrieving intent information. Intent setting functions also include functions for adding, updating, or removing expectations from an existing intent object as well as functions for retrieving information for specific expectations. Intent setting functions also include functions for adding context (e.g., inference data), updating or removing context from existing intents or expectations as well as functions for retrieving context from an intelligence component (e.g., local intelligence component 513 and 518 or domain intelligence components 528 and 533 of Figure 5). In some embodiments, intent API functions are handled by multiple components of a single system. For example, some operations are executed by an intent management component while other operations are executed by a control loop management component.

[0044] An intent setting request means that the intent is sent by an intent owner for operation by an intent handler. When the intent owner sends an intent setting request, the intent handler is expected to act and meet the requirements/expectations specified in intent request with the resources available to intent handler. For example, intent tester system 125, acting as intent owner, sends intent under test 108 to intent system under test 135, acting as intent handler 216. In response to receiving intent under test 108, intent system under test 135 executes intent test on target 220 according to the test requirements and network conditions specified by intent under test 108. For example, intent system under test 135 receives intent under test 108 including test requirements (e.g., test requirements 315 of Figure 3) and test conditions (test conditions 320 of Figure 3) and performs the intent test on target 220 based on the test requirements and test conditions.

[0045] Intent negotiation functions include functions communicating feasibility of requirements between the intent owner and intent handlers (e.g., intent expectations and parameters) as well as including functions indicating preference of solutions and outcomes. In some embodiments, intent tester system 125 sends intent under test 108 for indicating best values for some of the test requirements. In such embodiments, intent handler 216 is not expected to necessarily meet the requirements of intent under test 108 but intent handler 216 is expected to send intent test report 110 including the best values possible if intent handler 216 were to implement the requirements.

[0046] Intent reporting functions include functions that create and send intent reports to the intent owner according to expectations set by the intent owner in the original intent request. In some embodiments, intent functions send intent reports at core points in the intent lifecycle (e.g., acceptance, modification, or violation of an intent). In some embodiments, the intent owner sends reporting expectations in an intent request (e.g., intent under test 108). For example, intent under test 108 includes reporting requirements for when intent test report 110 should be provided by intent handler 216. For example, intent under test 108 includes a parameter indicating that intent handler 216 should send intent test report 110 at a specific time.

[0047] In some embodiments, in response to receiving start intent test request 225 from intent owner 105, intent tester system 125 determines systems under test 230. In some embodiments, start intent test request 225 includes an indication of which intent systems to test. For example, start intent test request 225 includes test environment information (e.g., test environment information 106 of Figure 1) specifying to test one specific system (e.g., intent system under test 135) or one specific domain of operation (e.g., resource operations 525 of Figure 5). In other embodiments, intent tester system 125 determines intent system under test 135 based on the test environment information and service requirements of start intent test request 225. For example, intent tester system 125 chooses intent system under test 135 from multiple managed intent systems based on intent system under test 135 satisfying a number of UEs service requirement of start intent test request 225. In some embodiments, intent tester system 125 determines systems under test 230 using intent manager profiles of downstream intent management systems. For example, each of the downstream intent management components include an intent manager profile acting as an inventory of all intent management functions available to that intent management component. For example, the intent management profile includes the intent manager scope, supported interface operations, supported serialization formats, and supported models. The intent manager scope describes the scope of responsibilities for that intent manager. In one example, the intent manager scope is slice management. The supported interface operations describes the procedures and functions that are supported by the intent manager. In one example, the supported interface operations include test, probe, best, and proposal. The supported serialization formats describes which formats are available for encoding intent requests and intent reports on the intent interface for the intent manager. In one embodiment, the supported serialization formats include turtle, extensible markup language (XML), and JavaScript Object Notation (JSON). The supported models describe which intent models are understood by the intent manager and which intent models can be used in intent requests and intent reports. In some embodiments, an intent owner (e.g., intent tester system 125) retrieves the intent management profile for an intent handler (e.g., intent handler 216) before invoking the intent API (e.g., sending intent under test 108).

[0048] In some embodiments, in response to receiving start intent test request 225, intent tester system 125 generates set of intents under test 235. For example, intent tester system 125 uses the service requirements and the test environment information of start intent test request 225 received from intent owner 105 and translates them into a set of intents under test. Each of the intents under test of the set of intents under test is an intent object with test requirements (e.g., test requirements 315) and network conditions (e.g., test conditions 320). In some embodiments, intent tester system 125 translates the inputs of start intent test request 225 into a set of intents under test using a script and/or policy. For example, certain service requirements of start intent test request 225 may pair with test requirements for the set of intents under test. In one embodiment, start intent test request 225 includes a service requirement for a minimum data throughput of 1Mbps. In such an embodiment, intent tester system 125 uses a script and/or policy to generate a test requirement for an intent under test for a minimum data throughput of 1Mbps. In some embodiments, intent tester system 125 translates the inputs of start intent test request 225 into a set of intents under test using a trained machine learning model. For example, as described in further detail with reference to Figure 4, intent tester system 125 translates the inputs of start intent test request 225 into a set of intents under test using a machine learning model trained with datasets that represent the submitted requirements of operators (e.g., intent owners) and corresponding intents.

[0049] Intent tester system 125 sends intent under test 108 of the set of intents under test to intent handler 216 of intent system under test 135. For example, intent tester system 125 sends intent under test 108 including test requirements and network conditions as explained with reference to Figure 1. In some embodiments, in response to receiving an intent under test (e.g., intent under test 108), the intent system under test (e.g., intent system under test 135) determines that it cannot complete or perform the test. For example, the intent system under test determines that the test requirements need to be further refined into technology-specific requirements (e.g., from business requirements to service or resource-specific requirements) and/or determines that the intent system under test does not have access to required resources for the test. In such embodiments, the intent system under test act as an intent owner for a new intent under test and decomposes the received intent under test into a set of intents under test sent to intent subsystems under test. In some embodiments, this process is repeated until the correct level of requirements and the required resources are reached.

[0050] Intent handler 216 of intent system under test 135 receives intent under test 108 and determines target of intent under test 245. For example, intent handler 216 determines target 220 as the target of intent under test 108. In some embodiments, intent under test 108 includes an indication of which targets to test. For example, intent under test 108 includes test conditions (e.g., test conditions 320 of Figure 1) specifying to test one specific target (e.g., target 220). In other embodiments, intent system under test determines target of intent under test based on the test conditions and test requirements of intent under test 108. For example, intent system under test 135 chooses target 220 from multiple managed targets based on target 220 satisfying test requirements of intent under test 108. Intent handler 216 executes test 250 on target 220. In some embodiments, target 220 is a network device (e.g., network element 540 of a real network (e.g., network 145). In some embodiments, model 222 is a machine learning model trained to simulate a network element of a real network. Further details regarding executing a test are explained with reference to Figure 1.

[0051] In some embodiments, in response to receiving intent under test 108 from intent tester system 125, intent system under test 135 sends a test received message (not illustrated). In some embodiments, intent tester system 125 sends intent under test 108 again to the same intent system under test 135 if intent tester system 125 does not receive the received message. In other embodiments, intent tester system 125 sends intent under test 108 to a different intent system under test 135 if intent tester system 125 does not receive the received message. In some embodiments, intent system under test 135 includes an intent management component operating as intent handler 216, a control loop management component, and a domain intelligence component. For example, intent system under test 135 is implemented at a resource operations layer (e.g., resource operations 525) and includes intent management component 526 (operating as intent handler 216), control loop management component 527, and domain intelligence component 528. In such embodiments, intent management component 526 sends control loop management component 527 a handle intent directive to handle test requirements and/or test conditions received in intent under test 108. Control loop management component 527 translates the test requirements and/or test conditions of intent under test 108 into service or network tests executes the service or network tests (e.g., execute test 250) on target 220. For example, intent management component 526 acting as intent handler 216 uses the knowledge from domain intelligence component 528 to translate test requirements of intent under test 108 into actions which are sent to target 220 as execute test 250. In some embodiments, control loop management component 527 uses reasoning procedures (such as by using a domain intelligence component 528) such as reasoning inference machine learning model inference, and/or machine reasoning to translate the requirements of intent under test into execute test 250. Control loop management component 527 executes the translated action (e.g., execute test 250) on target 220. [0052] In some embodiments, intent handler 216 collects test results 255 from target 220 in response to executing test 250 on target 220. For example, intent handler 216 collects test results reflecting how target 220 actually performed during the test or how model 222 predicted that a network element would perform. In some embodiments, when using a network digital twin (e.g., virtual network 155 of Figure 1), test results 255 is similar to data collected during regular intent fulfillment and assurance (e.g., data collected in response to regular intent operations). For example, intent system under test 135 receives data from target 220 and includes the data in intent test report 110. In other embodiments, intent system under test 135 collects test results 255 and derives outcomes to include in intent test report 110 based on test results 255. For example, intent system under test 135 receives test results 255 from target 220 and compiles test results 255 with test results from other targets and/or performs other processing operations to derive test outcomes based on the test results. These test outcomes, for example, can include test results for intent under test 108 (e.g., test results 325 of Figure 3) and network conditions for intent under test 108 (e.g., network conditions 330 of Figure 3). In some embodiments, intent system under test has the autonomy to determine how to execute intent tests (e.g., execute test 250), collect data (e.g., collect test results 255), and report back to intent tester system 125 (e.g., send intent test report 110).

[0053] Intent handler 216 collects test results 255 and generates intent test report 110 using collected test results 255. In some embodiments, intent handler 216 uses test results 255 to generate intent test report 110 at the same level of abstraction as the received intent under test 108. For example, in embodiments where intent handler 216 is implemented at a resource operations layer (e.g., resource operations 525 of Figure 5), intent handler 216 implemented by intent management component 526 can send test results 255 to domain intelligence component 528 which translates test results 255 based on the level of abstraction from intent under test 108. In some embodiments, intent test report 110 includes test results (e.g., test results 325 of Figure 3) and network conditions (e.g., network conditions 330 of Figure 3). Further details regarding intent test report 110 are described with references to Figures 1 and 3.

[0054] Intent tester system 125 receives intent test report 110 and generates intent test report 265. In some embodiments, intent tester system 125 receives multiple intent reports (including intent test report 110) from multiple intent systems under test (including intent system under test 135). In such embodiments, intent tester system 125 can accumulate or otherwise combine the intent test reports to generate intent test report 265. In some embodiments, intent tester system 125 generate intent test report 265 at the same level of abstraction as the received start intent test request 225. For example, in embodiments where intent tester system 125 is implemented at a service operations layer (e.g., service operations 515 of Figure 5), intent tester system 125 implemented by intent management component 516 can send the received intent test reports (including intent test report 110) to local intelligence component 518 which translates the received intent test reports based on the level of abstraction from start intent test request 225. In one example, if start intent test request 225 includes service requirements and/or test environment information in a natural language, intent tester system 125 provides intent test report 265 with responses in a natural language. [0055] In one exemplary embodiment, as shown in Figure 3, intent under test 108 includes test requirements 315 and test conditions 320. For example, intent under test 108 includes test requirements 315 indicating a number range of UEs (e.g., between 100 and 10,000), a geographical coverage area, a data throughput requirement, an energy consumption requirement, and a data privacy requirement. As explained above, some of the requirements of test requirements 315 can be framed as best values. In such embodiments, the intent system under test 135 would report back the best value it can achieve for that requirement given the constraints of test requirements 315 and test conditions 320. Continuing with the example, intent under test 108 includes test conditions 320 indicating that intent system should be tested during business hours and during major sport events.

[0056] In another exemplary embodiment, as shown in Figure 3, intent test report 110 includes test results 325 and network conditions 330. For example, test results 325 includes reported results for the intent under test 108 based on received test requirements 315. Test results 325 therefore includes, in continuing with the above example, the number range of UEs during the test, the achieved geographical coverage area during the test, the data throughput achieved during the test, the energy consumed during the test, and whether data privacy compliance was achieved. In some embodiments, in addition or instead of including actual parameters (e.g., the exact data throughput number), test results 325 includes an indication of whether test requirements 315 were met or not. For example, test results 325 includes an indication that the required data throughput of test requirements 315 was met but does not include an actual calculated data throughput value.

[0057] As shown in Figure 3, intent tester system 125 may be implemented on a first electronic device 305 and intent system under test 135 may be implemented on a second electronic device 310. Intent under test 108 and intent test report 110 may therefore be messages sent between first electronic device 305 and second electronic device 310.

[0058] Figure 4 illustrates exemplary intent tester system, according to some embodiments of the invention. As shown in Figure 4, intent tester system 125 includes intent under test generator 405. Intent under test generator 405 includes model inference 415, machine learning models 425, and machine learning model trainer 435. Model inference 415 receives start intent test request 225 and generates intent under test 108. For example, model inference 415 receives start intent test request 225 including service requirements and test environment information from intent owner 105 and generates intent under test 108 including test requirements and test conditions. In some embodiments, as shown in Figure 4, model inference 415 uses machine learning models 425 trained by machine learning model trainer 435. Machine learning model trainer 435 collects and/or creates training dataset 445 including intent test request training dataset 446 and intent under test training dataset 448. In some embodiments, training dataset 445 maps different intent test requests (e.g., service requirements and test environment information) with correctly generated intents under test. [0059] In some embodiments, machine learning model trainer 435 trains machine learning models 425 using training dataset 445 such that model inference 415 can use machine learning models 425 to correctly translate between start intent requests and intents under test. In some embodiments, each model of machine learning model trainer 435 is specific to a certain type of intent system under test. For example, machine learning model trainer 435 includes models for each specific domain of an autonomous network (e.g., autonomous network 500 of Figure 5).

[0060] In some embodiments, test generator 405 includes a conversion layer that extracts features and corresponding values from start intent test request 225 that are understandable for machine learning models 425. For example, test generator 405 includes a conversion layer that translates the service requirements and test environment information specified by start intent test request 225 into data that can be input into machine learning models 425. [0061] Figure 5 illustrates an exemplary network architecture of autonomous network 500, according to some embodiments of the invention. As shown in Figure 5, exemplary network architecture of autonomous network 500 may include intent owner 505, business operations 510, service operations 515, network intelligence component 520, resource operations 525 and network elements 540 and 545. Business operations 510, service operations 515, network intelligence component 520, and resource operations 525 are implemented on one or more general -purpose control plane devices, such as later described with reference to general- purpose control plane device 804 of Figure 8. Such general-purpose control plane devices may therefore implement one or both of first electronic device 305 and second electronic device 310 of Figure 3. Network elements 540 and 545 may be physical or virtual network elements, such as later described with reference to virtual network elements 730 A or 760 A of Figure 7A. In some embodiments, one or more of business operations 510, service operations 515, network intelligence component 520, and resource operations 525 are included in the same general-purpose control plane device. In some embodiments, site inference includes other artificial intelligence techniques such as reasoning inference and/or machine reasoning. [0062] Intent functions (e.g., intent under test 108 and intent test report 110) may occur at multiple levels of exemplary network architecture of autonomous network 500. For example, intent management component 511 is the intent owner and intent management component 516 is the intent handler in one intent interaction. In another example, intent management component 516 is the intent owner and intent management component 526 is the intent handler. In still another embodiment, intent management component 526 is the intent owner and intent management component 531 is the intent handler. The interactions explained in Figures 1-4 (including intents under test and intent test reports) can therefore occur between and among different devices within exemplary network architecture of autonomous network 500.

[0063] In some embodiments, exemplary network architecture of autonomous network 500 also includes autonomous domains, such as autonomous domains 550 and 555. Although only two autonomous domains, 550 and 555, are illustrated, any number of autonomous domains may be implemented. Additionally, although illustrated only at the resource operational layer, autonomous domains may exist at any operational layer and may even span multiple operational layers. For example, an autonomous domain may include business operations 510 and service operations 515. In some embodiments, autonomous domains are organized in a layered, peering, or orthogonal way to promote virtualization of an autonomous network over the physical infrastructure. For example, network elements can concurrently belong to different autonomous domains. In some embodiments, autonomous domains 550 and 555 are determined by a virtualization layer, such as virtualization layer 754 of Figure 7A. In some embodiments, autonomous domains 550 and 555 are self-governing virtual network elements including self-X capabilities, such as self-optimization, self-healing, and self-protection. For example, control loop management component 527 and/or control loop management component 532 of autonomous domain 550 and/or 555 enables and executes self-X capabilities for autonomous domain 550.

[0064] Business operations 510 includes intent management component 511, control loop management component 512, and local intelligence component 513. In some embodiments intent management component 511 implements external intent application program interface (API) interactions between components in an autonomous network. For example, exemplary network architecture of autonomous network 500 is an autonomous network operating using an intent-driven interface enabling interactions between intent owners and intent handlers for intent stages include setting intent, reporting on intent, negotiating intent, and profile handling. For example, an intent is a formal specification of expectations including requirements, goals, and constraints for a system, such as exemplary network architecture of autonomous network 500. An intent can be specific to an operational layer (such as service operations 515) or may be more broadly applied to multiple operational layers (such as business operation 510 and service operation 515). Intent-driven interfaces include an intent owner (e.g., intent owner 505) which sets the intent (i.e., requirements, goals, and constraints) and an intent handler which receives the intent and determines whether it has resources to fulfill the intent. In some embodiments, intent management component 511 can act as both intent handler and intent owner. For example, intent management component 511 can both receive and reply to an intent from intent owner 505 and send an intent to an intent handler, such as intent management component 516.

[0065] The autonomous network 500 includes three operational layers: a business operational layer including business operations 510, a service operational layer including service operations 515, and a resource operational layer including resource operations 525 and network element 540. In some embodiments, each of the operational layers has the capability to run in a self-operating mode such that details of network implementations, operations, and functions within the operational layer are not shown to devices outside of the operational layer.

[0066] Service operations 515 includes intent management component 516, control loop management component 517, and local intelligence component 518. In some embodiments intent management component 516 implements external intent application program interface (API) interactions between components in an autonomous network.

[0067] Service operations 515 includes intent management component 516, control loop management component 517, and local intelligence component 518. In some embodiments intent management component 516 implements external intent application program interface (API) interactions between components in an autonomous network.

[0068] Resource operations 525 includes intent management components 526 and 531, control loop management components 527 and 532, and domain intelligence components 528 and 533. In some embodiments intent management components 526 and 531 implement external intent application program interface (API) interactions between components in an autonomous network.

[0069] In some embodiments, network intelligence component 520 of Figure 5 provides intelligence services for all three operational layers. For example, network intelligence component 520 includes model training component 521 which trains machine learning models (such as machine learning model 222 of Figure 2 and/or machine learning models 425 of Figure 4) based on data in network intelligence component 520 and sends the trained models to local intelligence component 513 and 518 and domain intelligence components 528 and 533. In some embodiments, network intelligence component 520 receives offline data to tune its training algorithms. For example, network intelligence component 520 receives data about the efficacy of the trained machine learning models and updates model training component 521 based on the data. In some embodiments, machine learning model trainer 435 is implemented by model training component 521.

[0070] Figure 6 is a flow diagram of an example method to test intents in an intent management system. The operations in the flow diagram will be described with reference to the exemplary embodiments of the other figures. However, it should be understood that the operations of the flow diagram can be performed by embodiments of the invention other than those discussed with reference to the other figures, and the embodiments of the invention discussed with reference to these other figures can perform operations different than those discussed with reference to the flow diagram. Method 600 may be implemented by one or more of intent tester system 125, intent system under test 135, and/or target 220 of Figure 2. [0071] At operation 605, the processing device receives, at an intent handler implemented by a first electronic device (e.g., first electronic device 305) a request (e.g., start intent test request 225) from an intent owner (e.g., intent owner 105) to implement an intent test. The request includes service requirements and test environment information. For example, intent tester system 125 receives start intent test request 225 including service requirements 104 and test environment information 106 from intent owner 105. Further details with regard to the operations of receiving and processing an intent test request are explained with reference to Figures 1-4.

[0072] At operation 610, the processing device determines a set of intent systems under test using the service requirements. For example, intent tester system 125 determines a set of intent systems under test including intent system under test 135 using service requirements 104 of start intent test request 225. In some embodiments, intent tester system 125 determines the set of intent systems under test based on the service that is requested (e.g., service requirements 104). Further details with regard to the operations of determining a set of intent systems under test are explained with reference to Figures 1-4.

[0073] At operation 615, the processing device generates an intent under test for each of the set of intent systems using the service requirements and the test environment information. For example, intent tester system 125 generates intent under test 108 for intent system under test 135 using service requirements 104 and test environment information 106. Further details with regard to the operations of generating an intent under test are explained with reference to Figures 1-4.

[0074] At operation 620, the processing device sends to the set of intent systems under test, the intents under test. For example, intent tester system 125 sends intent under test 108 to intent system under test 135. In some embodiments, intent under test includes test requirements and test conditions (e.g., test requirements 315 and test conditions 320 of Figure 3). Further details with regard to the operations of sending an intent under test are explained with reference to Figures 1-4.

[0075] At operation 625, the processing device receives from the set of intent systems under test, intent test reports in response to sending the intents under test. The intent test report includes information above the set of intent systems under test implementing the intents under test. For example, intent tester system 125 receives intent test report 110 from intent system under test 135 in response to intent under test 108. In some embodiments, intent test report 110 includes test results 325 and network conditions 330. Further details with regard to the operations of receiving intent test reports are explained with reference to Figures 1-4.

[0076] At operation 630, the processing device generates, in response to the request to implement the intent test using the intent test reports for the intent systems under test, a response to the request to implement the intent test. For example, intent tester system 125 generates intent test report 265 to send to intent owner 105 in response to start intent test request 225. Further details with regard to the operations of generating a response to the request to implement the test are explained with reference to Figures 1-4.

[0077] At operation 635, the processing device sends, to the intent owner, the response. For example, intent tester system 125 sends intent test report 265 to intent owner 105. Further details with regard to the operations of sending a response to the request to implement the test are explained with reference to Figures 1-4.

[0078] An electronic device stores and transmits (internally and/or with other electronic devices over a network, such as autonomous network 500) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and/or data using machine-readable media (also called computer-readable media or a memory unit), such as machine-readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device, such as first electronic device 305 and/or second electronic device 310, (e.g., a computer) includes hardware and software, such as a set of one or more processors (e.g., wherein a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array, other electronic circuitry, a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and/or to store data. For instance, an electronic device may include non-volatile memory containing the code since the non-volatile memory can persist code/data even when the electronic device is turned off (when power is removed), and while the electronic device is turned on that part of the code that is to be executed by the processor(s) of that electronic device is typically copied from the slower nonvolatile memory into volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of that electronic device. Typical electronic devices also include a set of one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and/or receive code and/or data using propagating signals) with other electronic devices. For example, the set of physical NIs (or the set of physical NI(s) in combination with the set of processors executing code) may perform any formatting, coding, or translating to allow the electronic device to send and receive data whether over a wired and/or a wireless connection. In some embodiments, a physical NI may comprise radio circuitry capable of receiving data from other electronic devices over a wireless connection and/or sending data out to other devices via a wireless connection. This radio circuitry may include transmitter(s), receiver(s), and/or transceiver s) suitable for radiofrequency communication. The radio circuitry may convert digital data into a radio signal having the appropriate parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signal may then be transmitted via antennas to the appropriate recipient(s). In some embodiments, the set of physical NI(s) may comprise network interface controller(s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter. The NIC(s) may facilitate in connecting the electronic device to other electronic devices allowing them to communicate via wire through plugging in a cable to a physical port connected to a NIC. One or more parts of an embodiment of the invention may be implemented using different combinations of software, firmware, and/or hardware.

[0079] A network device (ND) is an electronic device (e.g., first electronic device 305 or second electronic device 310) that communicatively interconnects other electronic devices on the network (e.g., other network devices, end-user devices). Some network devices are “multiple services network devices” that provide support for multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and/or subscriber management), and/or provide support for multiple application services (e.g., data, voice, and video).

[0080] Figure 7A illustrates connectivity between network devices (NDs) within an exemplary network, such as autonomous network 500 as well as three exemplary implementations of the NDs, according to some embodiments of the invention. Figure 7A shows NDs 700A-H, and their connectivity by way of lines between 700A-700B, 700B-700C, 700C-700D, 700D-700E, 700E-700F, 700F-700G, and 700A-700G, as well as between 700H and each of 700A, 700C, 700D, and 700G. These NDs are physical devices, and the connectivity between these NDs can be wireless or wired (often referred to as a link). An additional line extending from NDs 700A, 700E, and 700F illustrates that these NDs act as ingress and egress points for the network (and thus, these NDs are sometimes referred to as edge NDs, while the other NDs may be called core NDs).

[0081] Two of the exemplary ND implementations in Figure 7A are: 1) a special-purpose network device 702 that uses custom application-specific integrated-circuits (ASICs) and a special-purpose operating system (OS); and 2) a general -purpose network device 704 that uses common off-the-shelf (COTS) processors and a standard OS.

[0082] The special -purpose network device 702 includes networking hardware 710 comprising a set of one or more processor(s) 712, forwarding resource(s) 714 (which typically include one or more ASICs and/or network processors), and physical network interfaces (NIs) 716 (through which network connections are made, such as those shown by the connectivity between NDs 700A-H), as well as non-transitory machine-readable storage media 718 having stored therein networking software 720. During operation, the networking software 720 may be executed by the networking hardware 710 to instantiate a set of one or more networking software instance(s) 722. Each of the networking software instance(s) 722, and that part of the networking hardware 710 that executes that network software instance (be it hardware dedicated to that networking software instance and/or time slices of hardware temporally shared by that networking software instance with others of the networking software instance(s) 722), form a separate virtual network element 730A-R. Each of the virtual network element(s) (VNEs) 730A-R includes a control communication and configuration module 732A-R (sometimes referred to as a local control module or control communication module) and forwarding table(s) 734A-R, such that a given virtual network element (e.g., 730A) includes the control communication and configuration module (e.g., 732A), a set of one or more forwarding table(s) (e.g., 734A), and that portion of the networking hardware 710 that executes the virtual network element (e.g., 730A, 540, or 545). In some embodiments, each of the operational layers of Figure 5 are implemented in separate virtual network elements (e.g., virtual network elements 730A-730R).

[0083] The special-purpose network device 702 is often physically and/or logically considered to include: 1) a ND control plane 724 (sometimes referred to as a control plane) comprising the processor(s) 712 that execute the control communication and configuration module(s) 732A-R; and 2) a ND forwarding plane 726 (sometimes referred to as a forwarding plane, a data plane, or a media plane) comprising the forwarding resource(s) 714 that utilize the forwarding table(s) 734A-R and the physical Nis 716. By way of example, where the ND is a router (or is implementing routing functionality), the ND control plane 724 (the processor(s) 712 executing the control communication and configuration module(s) 732A-R) is typically responsible for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) and storing that routing information in the forwarding table(s) 734A-R, and the ND forwarding plane 726 is responsible for receiving that data on the physical Nis 716 and forwarding that data out the appropriate ones of the physical Nis 716 based on the forwarding table(s) 734A-R.

[0084] Figure 7B illustrates an exemplary way to implement the special-purpose network device 702 according to some embodiments of the invention. Figure 7B shows a specialpurpose network device including cards 738 (typically hot pluggable). While in some embodiments the cards 738 are of two types (one or more that operate as the ND forwarding plane 726 (sometimes called line cards), and one or more that operate to implement the ND control plane 724 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and/or include additional card types (e.g., one additional type of card is called a service card, resource card, or multi-application card). A service card can provide specialized processing (e.g., for service operations 515 and/or Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway)). By way of example, a service card may be used to terminate IPsec tunnels and execute the attendant authentication and encryption algorithms. These cards are coupled together through one or more interconnect mechanisms illustrated as backplane 736 (e.g., a first full mesh coupling the line cards and a second full mesh coupling all of the cards). [0085] Returning to Figure 7A, the general-purpose network device 704 includes hardware 740 comprising a set of one or more processor(s) 742 (which are often COTS processors) and physical NIs 746, as well as non-transitory machine-readable storage media 748 having stored therein software 750. During operation, the processor(s) 742 execute the software 750 to instantiate one or more sets of one or more applications 764A-R. In some embodiments, the one or more applications 764A-R are associated with the delivery of a service. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment the virtualization layer 754 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 762A-R called software containers that may each be used to execute one (or more) of the sets of applications 764A-R; where the multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run; and where the set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes. In another such alternative embodiment the virtualization layer 754 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 764A-R is run on top of a guest operating system within an instance 762A-R called a virtual machine (which may in some cases be considered a tightly isolated form of software container) that is run on top of the hypervisor - the guest operating system and application may not know they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or through para-virtualization the operating system and/or application may be aware of the presence of virtualization for optimization purposes. In yet other alternative embodiments, one, some or all of the applications are implemented as unikemel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers/libraries of OS services) that provide the particular OS services needed by the application. As a unikemel can be implemented to run directly on hardware 740, directly on a hypervisor (in which case the unikemel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented fully with unikernels running directly on a hypervisor represented by virtualization layer 754, unikemels running within software containers represented by instances 762A-R, or as a combination of unikernels and the abovedescribed techniques (e.g., unikemels and virtual machines both run directly on a hypervisor, unikemels and sets of applications that are run in different software containers).

[0086] The instantiation of the one or more sets of one or more applications 764A-R, as well as virtualization if implemented, are collectively referred to as software instance(s) 752. Each set of applications 764A-R, corresponding virtualization construct (e.g., instance 762A- R) if implemented, and that part of the hardware 740 that executes them (be it hardware dedicated to that execution and/or time slices of hardware temporally shared), forms a separate virtual network element(s) 760A-R (e.g., network elements 540 and 545).

[0087] The virtual network element(s) 760 A-R perform similar functionality to the virtual network element(s) 730A-R - e.g., similar to the control communication and configuration module(s) 732A and forwarding table(s) 734A (this virtualization of the hardware 740 is sometimes referred to as network function virtualization (NFV)). Thus, NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which could be located in Data centers, NDs, and customer premise equipment (CPE). While embodiments of the invention are illustrated with each instance 762 A-R corresponding to one VNE 760 A-R, alternative embodiments may implement this correspondence at a finer level granularity (e.g., line card virtual machines virtualize line cards, control card virtual machine virtualize control cards, etc.); it should be understood that the techniques described herein with reference to a correspondence of instances 762A-R to VNEs also apply to embodiments where such a finer level of granularity and/or unikemels are used.

[0088] In certain embodiments, the virtualization layer 754 includes a virtual switch that provides similar forwarding services as a physical Ethernet switch. Specifically, this virtual switch forwards traffic between instances 762A-R and the physical NI(s) 746, as well as optionally between the instances 762A-R; in addition, this virtual switch may enforce network isolation between the VNEs 760A-R that by policy are not permitted to communicate with each other (e.g., by honoring virtual local area networks (VLANs)).

[0089] The third exemplary ND implementation in Figure 7Ais a hybrid network device 706, which includes both custom ASICs/special-purpose OS and COTS processors/standard OS in a single ND or a single card within an ND. In certain embodiments of such a hybrid network device, a platform VM (i.e., a VM that that implements the functionality of the special-purpose network device 702) could provide for para-virtualization to the networking hardware present in the hybrid network device 706.

[0090] Regardless of the above exemplary implementations of an ND, when a single one of multiple VNEs implemented by an ND is being considered (e.g., only one of the VNEs is part of a given virtual network) or where only a single VNE is currently being implemented by an ND, the shortened term network element (NE) is sometimes used to refer to that VNE (e.g., network elements 540 and 545). Also, in all of the above exemplary implementations, each of the VNEs (e.g., VNE(s) 730A-R, VNEs 760A-R, and those in the hybrid network device 706) receives data on the physical NIs (e.g., 716, 746) and forwards that data out the appropriate ones of the physical NIs (e.g., 716, 746). For example, a VNE implementing IP router functionality forwards IP packets on the basis of some of the IP header information in the IP packet; where IP header information includes source IP address, destination IP address, source port, destination port (where “source port” and “destination port” refer herein to protocol ports, as opposed to physical ports of a ND), transport protocol (e.g., user datagram protocol (UDP), Transmission Control Protocol (TCP), and differentiated services code point (DSCP) values.

[0091] Figure 7C illustrates various exemplary ways in which VNEs may be coupled according to some embodiments of the invention. Figure 7C shows VNEs 770A.1-770A.P (and optionally VNEs 770A.Q-770A.R) implemented in ND 700A and VNE 770H.1 in ND 700H. In Figure 7C, VNEs 770A.1-P are separate from each other in the sense that they can receive packets from outside ND 700A and forward packets outside of ND 700A; VNE 770A.1 is coupled with VNE 770H.1, and thus they communicate packets between their respective NDs; VNE 770A.2-770A.3 may optionally forward packets between themselves without forwarding them outside of the ND 700A; and VNE 770A.P may optionally be the first in a chain of VNEs that includes VNE 770A.Q followed by VNE 770A.R (this is sometimes referred to as dynamic service chaining, where each of the VNEs in the series of VNEs provides a different service - e.g., one or more layer 4-7 network services). While Figure 7C illustrates various exemplary relationships between the VNEs, alternative embodiments may support other relationships (e.g., more/fewer VNEs, more/fewer dynamic service chains, multiple different dynamic service chains with some common VNEs and some different VNEs).

[0092] The NDs of Figure 7A, for example, may form part of the Internet or a private network; and other electronic devices (not shown; such as end user devices including workstations, laptops, netbooks, tablets, palm tops, mobile phones, smartphones, phablets, multimedia phones, Voice Over Internet Protocol (VOIP) phones, terminals, portable media players, GPS units, wearable devices, gaming systems, set-top boxes, Internet enabled household appliances) may be coupled to the network (directly or through other networks such as access networks) to communicate over the network (e.g., the Internet or virtual private networks (VPNs) overlaid on (e.g., tunneled through) the Internet) with each other (directly or through servers) and/or access content and/or services. Such content and/or services are typically provided by one or more servers (not shown) belonging to a service/content provider or one or more end user devices (not shown) participating in a peer-to-peer (P2P) service, and may include, for example, public webpages (e.g., free content, store fronts, search services), private webpages (e.g., usemame/password accessed webpages providing email services), and/or corporate networks over VPNs. For instance, end user devices may be coupled (e.g., through customer premise equipment coupled to an access network (wired or wirelessly)) to edge NDs, which are coupled (e.g., through one or more core NDs) to other edge NDs, which are coupled to electronic devices acting as servers. However, through compute and storage virtualization, one or more of the electronic devices operating as the NDs in Figure 7A may also host one or more such servers (e.g., in the case of the general-purpose network device 704, one or more of the software instances 762A-R may operate as servers; the same would be true for the hybrid network device 706; in the case of the special -purpose network device 702, one or more such servers could also be run on a virtualization layer executed by the processor(s) 712); in which case the servers are said to be co-located with the VNEs of that ND. [0093] A virtual network is a logical abstraction of a physical network (such as that in Figure 7A) that provides network services (e.g., L2 and/or L3 services). A virtual network can be implemented as an overlay network (sometimes referred to as a network virtualization overlay) that provides network services (e.g., layer 2 (L2, data link layer) and/or layer 3 (L3, network layer) services) over an underlay network (e.g., an L3 network, such as an Internet Protocol (IP) network that uses tunnels (e.g., generic routing encapsulation (GRE), layer 2 tunneling protocol (L2TP), IPSec) to create the overlay network).

[0094] A network virtualization edge (NVE) sits at the edge of the underlay network and participates in implementing the network virtualization; the network -facing side of the NVE uses the underlay network to tunnel frames to and from other NVEs; the outward -facing side of the NVE sends and receives data to and from systems outside the network. A virtual network instance (VNI) is a specific instance of a virtual network on a NVE (e.g., a NE/VNE on an ND, a part of a NE/VNE on a ND where that NE/VNE is divided into multiple VNEs through emulation); one or more VNIs can be instantiated on an NVE (e.g., as different VNEs on an ND). A virtual access point (VAP) is a logical connection point on the NVE for connecting external systems to a virtual network; a VAP can be physical or virtual ports identified through logical interface identifiers (e.g., a VLAN ID).

[0095] Examples of network services include: 1) an Ethernet LAN emulation service (an Ethernet-based multipoint service similar to an Internet Engineering Task Force (IETF) Multiprotocol Label Switching (MPLS) or Ethernet VPN (EVPN) service) in which external systems are interconnected across the network by a LAN environment over the underlay network (e.g., an NVE provides separate L2 VNIs (virtual switching instances) for different such virtual networks, and L3 (e.g., IP/MPLS) tunneling encapsulation across the underlay network); and 2) a virtualized IP forwarding service (similar to IETF IP VPN (e.g., Border Gateway Protocol (BGP)/MPLS IPVPN) from a service definition perspective) in which external systems are interconnected across the network by an L3 environment over the underlay network (e.g., an NVE provides separate L3 VNIs (forwarding and routing instances) for different such virtual networks, and L3 (e.g., IP/MPLS) tunneling encapsulation across the underlay network)). Network services may also include quality of service capabilities (e.g., traffic classification marking, traffic conditioning and scheduling), security capabilities (e.g., filters to protect customer premises from network - originated attacks, to avoid malformed route announcements), and management capabilities (e.g., full detection and processing).

[0096] Fig. 7D illustrates a network with a single network element on each of the NDs of Figure 7A, and within this straightforward approach contrasts a traditional distributed approach (commonly used by traditional routers) with a centralized approach for maintaining reachability and forwarding information (also called network control), according to some embodiments of the invention. Specifically, Figure 7D illustrates network elements (NEs) 770A-H with the same connectivity as the NDs 700A-H of Figure 7A.

[0097] Figure 7D illustrates that the distributed approach 772 distributes responsibility for generating the reachability and forwarding information across the NEs 770A-H; in other words, the process of neighbor discovery and topology discovery is distributed.

[0098] For example, where the special-purpose network device 702 is used, the control communication and configuration module(s) 732A-R of the ND control plane 724 typically include a reachability and forwarding information module to implement one or more routing protocols (e.g., an exterior gateway protocol such as Border Gateway Protocol (BGP), Interior Gateway Protocol(s) (IGP) (e.g., Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), Routing Information Protocol (RIP), Label Distribution Protocol (LDP), Resource Reservation Protocol (RSVP) (including RSVP-Traffic Engineering (TE): Extensions to RSVP for LSP Tunnels and Generalized Multi-Protocol Label Switching (GMPLS) Signaling RSVP-TE)) that communicate with other NEs to exchange routes, and then selects those routes based on one or more routing metrics. Thus, the NEs 770A-H (e.g., the processor(s) 712 executing the control communication and configuration module(s) 732A- R) perform their responsibility for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) by distributively determining the reachability within the network and calculating their respective forwarding information. Routes and adjacencies are stored in one or more routing structures (e.g., Routing Information Base (RIB), Label Information Base (LIB), one or more adjacency structures) on the ND control plane 724. The ND control plane 724 programs the ND forwarding plane 726 with information (e.g., adjacency and route information) based on the routing structure(s). For example, the ND control plane 724 programs the adjacency and route information into one or more forwarding table(s) 734A-R (e.g., Forwarding Information Base (FIB), Label Forwarding Information Base (LFIB), and one or more adjacency structures) on the ND forwarding plane 726. For layer 2 forwarding, the ND can store one or more bridging tables that are used to forward data based on the layer 2 information in that data. While the above example uses the special-purpose network device 702, the same distributed approach 772 can be implemented on the general-purpose network device 704 and the hybrid network device 706.

[0099] Figure 7D illustrates that a centralized approach 774 (also known as software defined networking (SDN)) that decouples the system that makes decisions about where traffic is sent from the underlying systems that forwards traffic to the selected destination. The illustrated centralized approach 774 has the responsibility for the generation of reachability and forwarding information in a centralized control plane 776 (sometimes referred to as a SDN control module, controller, network controller, OpenFlow controller, SDN controller, control plane node, network virtualization authority, or management control entity), and thus the process of neighbor discovery and topology discovery is centralized. The centralized control plane 776 has a south bound interface 782 with a data plane 780 (sometime referred to the infrastructure layer, network forwarding plane, or forwarding plane (which should not be confused with a ND forwarding plane)) that includes the NEs 770A-H (sometimes referred to as switches, forwarding elements, data plane elements, or nodes). The centralized control plane 776 includes a network controller 778, which includes a centralized reachability and forwarding information module 779 that determines the reachability within the network and distributes the forwarding information to the NEs 770A-H of the data plane 780 over the south bound interface 782 (which may use the OpenFlow protocol). Thus, the network intelligence is centralized in the centralized control plane 776 executing on electronic devices that are typically separate from the NDs.

[00100] For example, where the special-purpose network device 702 is used in the data plane 780, each of the control communication and configuration module(s) 732A-R of the ND control plane 724 typically include a control agent that provides the VNE side of the south bound interface 782. In this case, the ND control plane 724 (the processor(s) 712 executing the control communication and configuration module(s) 732A-R) performs its responsibility for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) through the control agent communicating with the centralized control plane 776 to receive the forwarding information (and in some cases, the reachability information) from the centralized reachability and forwarding information module 779 (it should be understood that in some embodiments of the invention, the control communication and configuration module(s) 732A-R, in addition to communicating with the centralized control plane 776, may also play some role in determining reachability and/or calculating forwarding information - albeit less so than in the case of a distributed approach; such embodiments are generally considered to fall under the centralized approach 774, but may also be considered a hybrid approach).

[00101] While the above example uses the special-purpose network device 702, the same centralized approach 774 can be implemented with the general-purpose network device 704 (e.g., each of the VNE 760A-R performs its responsibility for controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) by communicating with the centralized control plane 776 to receive the forwarding information (and in some cases, the reachability information) from the centralized reachability and forwarding information module 779; it should be understood that in some embodiments of the invention, the VNEs 760A-R, in addition to communicating with the centralized control plane 776, may also play some role in determining reachability and/or calculating forwarding information - albeit less so than in the case of a distributed approach) and the hybrid network device 706. In fact, the use of SDN techniques can enhance the NFV techniques typically used in the general-purpose network device 704 or hybrid network device 706 implementations as NFV is able to support SDN by providing an infrastructure upon which the SDN software can be run, and NFV and SDN both aim to make use of commodity server hardware and physical switches.

[00102] Figure 7D also shows that the centralized control plane 776 has a north bound interface 784 to an application layer 786, in which resides application(s) 788. The centralized control plane 776 has the ability to form virtual networks 792 (sometimes referred to as a logical forwarding plane, network services, or overlay networks (with the NEs 770A-H of the data plane 780 being the underlay network)) for the application(s) 788. Thus, the centralized control plane 776 maintains a global view of all NDs and configured NEs/VNEs, and it maps the virtual networks to the underlying NDs efficiently (including maintaining these mappings as the physical network changes either through hardware (ND, link, or ND component) failure, addition, or removal).

[00103] While Figure 7D shows the distributed approach 772 separate from the centralized approach 774, the effort of network control may be distributed differently or the two combined in certain embodiments of the invention. For example: 1) embodiments may generally use the centralized approach (SDN) 774, but have certain functions delegated to the NEs (e.g., the distributed approach may be used to implement one or more of fault monitoring, performance monitoring, protection switching, and primitives for neighbor and/or topology discovery); or 2) embodiments of the invention may perform neighbor discovery and topology discovery via both the centralized control plane and the distributed protocols, and the results compared to raise exceptions where they do not agree. Such embodiments are generally considered to fall under the centralized approach 774 but may also be considered a hybrid approach.

[00104] While Figure 7D illustrates the simple case where each of the NDs 700A-H implements a single NE 770A-H, it should be understood that the network control approaches described with reference to Figure 7D also work for networks where one or more of the NDs 700A-H implement multiple VNEs (e.g., VNEs 730A-R, VNEs 760A-R, those in the hybrid network device 706). Alternatively, or in addition, the network controller 778 may also emulate the implementation of multiple VNEs in a single ND. Specifically, instead of (or in addition to) implementing multiple VNEs in a single ND, the network controller 778 may present the implementation of a VNE/NE in a single ND as multiple VNEs in the virtual networks 792 (all in the same one of the virtual network(s) 792, each in different ones of the virtual network(s) 792, or some combination). For example, the network controller 778 may cause an ND to implement a single VNE (a NE) in the underlay network, and then logically divide up the resources of that NE within the centralized control plane 776 to present different VNEs in the virtual network(s) 792 (where these different VNEs in the overlay networks are sharing the resources of the single VNE/NE implementation on the ND in the underlay network).

[00105] On the other hand, Figures 7E and 7F respectively illustrate exemplary abstractions of NEs and VNEs that the network controller 778 may present as part of different ones of the virtual networks 792. Figure 7E illustrates the simple case of where each of the NDs 700A-H implements a single NE 770A-H (see Figure 7D), but the centralized control plane 776 has abstracted multiple of the NEs in different NDs (the NEs 770A-C and G-H) into (to represent) a single NE 7701 in one of the virtual network(s) 792 of Figure 7D, according to some embodiments of the invention. Figure 7E shows that in this virtual network, the NE 7701 is coupled to NE 770D and 770F, which are both still coupled to NE 770E.

[00106] Figure 7F illustrates a case where multiple VNEs (VNE 770A.1 and VNE 770H.1) are implemented on different NDs (ND 700A and ND 700H) and are coupled to each other, and where the centralized control plane 776 has abstracted these multiple VNEs such that they appear as a single VNE 770T within one of the virtual networks 792 of Figure 7D, according to some embodiments of the invention. Thus, the abstraction of a NE or VNE can span multiple NDs.

[00107] While some embodiments of the invention implement the centralized control plane 776 as a single entity (e.g., a single instance of software running on a single electronic device), alternative embodiments may spread the functionality across multiple entities for redundancy and/or scalability purposes (e.g., multiple instances of software running on different electronic devices).

[00108] Similar to the network device implementations, the electronic device(s) running the centralized control plane 776, and thus the network controller 778 including the centralized reachability and forwarding information module 779, may be implemented a variety of ways (e.g., a special purpose device, a general-purpose (e.g., COTS) device, or hybrid device). These electronic device(s) would similarly include processor(s), a set of one or more physical NIs, and a non-transitory machine-readable storage medium having stored thereon the centralized control plane software. For instance, Figure 8 illustrates, a general-purpose control plane device 804 including hardware 840 comprising a set of one or more processor(s) 842 (which are often COTS processors) and physical NIs 846, as well as non-transitory machine- readable storage media 848 having stored therein centralized control plane (CCP) software 850.

[00109] In embodiments that use compute virtualization, the processor(s) 842 typically execute software to instantiate a virtualization layer 854 (e.g., in one embodiment the virtualization layer 854 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 862A-R called software containers (representing separate user spaces and also called virtualization engines, virtual private servers, or jails) that may each be used to execute a set of one or more applications; in another embodiment the virtualization layer 854 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and an application is run on top of a guest operating system within an instance 862A-R called a virtual machine (which in some cases may be considered a tightly isolated form of software container) that is run by the hypervisor ; in another embodiment, an application is implemented as a unikernel, which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers/libraries of OS services) that provide the particular OS services needed by the application, and the unikernel can run directly on hardware 840, directly on a hypervisor represented by virtualization layer 854 (in which case the unikernel is sometimes described as running within a LibOS virtual machine), or in a software container represented by one of instances 862A-R). Again, in embodiments where compute virtualization is used, during operation an instance of the CCP software 850 (illustrated as CCP instance 876A) is executed (e.g., within the instance 862A) on the virtualization layer 854. In embodiments where compute virtualization is not used, the CCP instance 876A is executed, as a unikernel or on top of a host operating system, on the “bare metal” general-purpose control plane device 804. The instantiation of the CCP instance 876A, as well as the virtualization layer 854 and instances 862A-R if implemented, are collectively referred to as software instance(s) 852.

[00110] In some embodiments, the CCP instance 876A includes a network controller instance 878. The network controller instance 878 includes a centralized reachability and forwarding information module instance 879 (which is a middleware layer providing the context of the network controller 778 to the operating system and communicating with the various NEs), and an CCP application layer 880 (sometimes referred to as an application layer) over the middleware layer (providing the intelligence required for various network operations such as protocols, network situational awareness, and user - interfaces). At a more abstract level, this CCP application layer 880 within the centralized control plane 776 works with virtual network view(s) (logical view(s) of the network) and the middleware layer provides the conversion from the virtual networks to the physical view. [00111] The centralized control plane 776 transmits relevant messages to the data plane 780 based on CCP application layer 880 calculations and middleware layer mapping for each flow. A flow may be defined as a set of packets whose headers match a given pattern of bits; in this sense, traditional IP forwarding is also flow-based forwarding where the flows are defined by the destination IP address for example, however, in other implementations, the given pattern of bits used for a flow definition may include more fields (e.g., 10 or more) in the packet headers. Different NDs/NEs/VNEs of the data plane 780 may receive different messages, and thus different forwarding information. The data plane 780 processes these messages and programs the appropriate flow information and corresponding actions in the forwarding tables (sometime referred to as flow tables) of the appropriate NE/VNEs, and then the NEs/VNEs map incoming packets to flows represented in the forwarding tables and forward packets based on the matches in the forwarding tables.

[00112] Standards such as OpenFlow define the protocols used for the messages, as well as a model for processing the packets. The model for processing packets includes header parsing, packet classification, and making forwarding decisions. Header parsing describes how to interpret a packet based upon a well-known set of protocols. Some protocol fields are used to build a match structure (or key) that will be used in packet classification (e.g., a first key field could be a source media access control (MAC) address, and a second key field could be a destination MAC address).

[00113] Packet classification involves executing a lookup in memory to classify the packet by determining which entry (also referred to as a forwarding table entry or flow entry) in the forwarding tables best matches the packet based upon the match structure, or key, of the forwarding table entries. It is possible that many flows represented in the forwarding table entries can correspond/match to a packet; in this case the system is typically configured to determine one forwarding table entry from the many according to a defined scheme (e.g., selecting a first forwarding table entry that is matched). Forwarding table entries include both a specific set of match criteria (a set of values or wildcards, or an indication of what portions of a packet should be compared to a particular value/values/wildcards, as defined by the matching capabilities - for specific fields in the packet header, or for some other packet content), and a set of one or more actions for the data plane to take on receiving a matching packet. For example, an action may be to push a header onto the packet, for the packet using a particular port, flood the packet, or simply drop the packet. Thus, a forwarding table entry for IPv4/IPv6 packets with a particular transmission control protocol (TCP) destination port could contain an action specifying that these packets should be dropped.

[00114] Making forwarding decisions and performing actions occurs, based upon the forwarding table entry identified during packet classification, by executing the set of actions identified in the matched forwarding table entry on the packet.

[00115] However, when an unknown packet (for example, a “missed packet” or a “match- miss” as used in OpenFlow parlance) arrives at the data plane 780, the packet (or a subset of the packet header and content) is typically forwarded to the centralized control plane 776. The centralized control plane 776 will then program forwarding table entries into the data plane 780 to accommodate packets belonging to the flow of the unknown packet. Once a specific forwarding table entry has been programmed into the data plane 780 by the centralized control plane 776, the next packet with matching credentials will match that forwarding table entry and take the set of actions associated with that matched entry.

[00116] A network interface (NI) may be physical or virtual; and in the context of IP, an interface address is an IP address assigned to a NI, be it a physical NI or virtual NI. A virtual NI may be associated with a physical NI, with another virtual interface, or stand on its own (e.g., a loopback interface, a point-to-point protocol interface). ANI (physical or virtual) may be numbered (a NI with an IP address) or unnumbered (a NI without an IP address). A loopback interface (and its loopback address) is a specific type of virtual NI (and IP address) of a NE/VNE (physical or virtual) often used for management purposes, where such an IP address is referred to as the nodal loopback address. The IP address(es) assigned to the NI(s) of a ND are referred to as IP addresses of that ND; at a more granular level, the IP address(es) assigned to NI(s) assigned to a NE/VNE implemented on a ND can be referred to as IP addresses of that NE/VNE.

[00117] Next hop selection by the routing system for a given destination may resolve to one path (that is, a routing protocol may generate one next hop on a shortest path); but if the routing system determines there are multiple viable next hops (that is, the routing protocol generated forwarding solution offers more than one next hop on a shortest path - multiple equal cost next hops), some additional criteria is used - for instance, in a connectionless network, Equal Cost Multi Path (ECMP) (also known as Equal Cost Multi Pathing, multipath forwarding and IP multipath) may be used (e.g., typical implementations use as the criteria particular header fields to ensure that the packets of a particular packet flow are always forwarded on the same next hop to preserve packet flow ordering). For purposes of multipath forwarding, a packet flow is defined as a set of packets that share an ordering constraint. As an example, the set of packets in a particular TCP transfer sequence need to arrive in order, else the TCP logic will interpret the out of order delivery as congestion and slow the TCP transfer rate down.

[00118] A Layer 3 (L3) Link Aggregation (LAG) link is a link directly connecting two NDs with multiple IP-addressed link paths (each link path is assigned a different IP address), and a load distribution decision across these different link paths is performed at the ND forwarding plane; in which case, a load distribution decision is made between the link paths.

[00119] Some NDs include functionality for authentication, authorization, and accounting (AAA) protocols (e.g., RADIUS (Remote Authentication Dial-In User Service), Diameter, and/or TACACS+ (Terminal Access Controller Access Control System Plus). AAA can be provided through a client/server model, where the AAA client is implemented on a ND and the AAA server can be implemented either locally on the ND or on a remote electronic device coupled with the ND. Authentication is the process of identifying and verifying a subscriber. For instance, a subscriber might be identified by a combination of a username and a password or through a unique key. Authorization determines what a subscriber can do after being authenticated, such as gaining access to certain electronic device information resources (e.g., through the use of access control policies). Accounting is recording user activity. By way of a summary example, end user devices may be coupled (e.g., through an access network) through an edge ND (supporting AAA processing) coupled to core NDs coupled to electronic devices implementing servers of service/content providers. AAA processing is performed to identify for a subscriber the subscriber record stored in the AAA server for that subscriber. A subscriber record includes a set of attributes (e.g., subscriber name, password, authentication information, access control information, rate-limiting information, policing information) used during processing of that subscriber’s traffic.

[00120] Certain NDs (e.g., certain edge NDs) internally represent end user devices (or sometimes customer premise equipment (CPE) such as a residential gateway (e.g., a router, modem) using subscriber circuits. A subscriber circuit uniquely identifies within the ND a subscriber session and typically exists for the lifetime of the session. Thus, a ND typically allocates a subscriber circuit when the subscriber connects to that ND, and correspondingly de-allocates that subscriber circuit when that subscriber disconnects. Each subscriber session represents a distinguishable flow of packets communicated between the ND and an end user device (or sometimes CPE such as a residential gateway or modem) using a protocol, such as the point-to-point protocol over another protocol (PPPoX) (e.g., where X is Ethernet or Asynchronous Transfer Mode (ATM)), Ethernet, 802. IQ Virtual LAN (VLAN), Internet Protocol, or ATM). A subscriber session can be initiated using a variety of mechanisms (e.g., manual provisioning a dynamic host configuration protocol (DHCP), DHCP/client-less internet protocol service (CLIPS) or Media Access Control (MAC) address tracking). For example, the point-to-point protocol (PPP) is commonly used for digital subscriber line (DSL) services and requires installation of a PPP client that enables the subscriber to enter a username and a password, which in turn may be used to select a subscriber record. When DHCP is used (e.g., for cable modem services), a username typically is not provided; but in such situations other information (e.g., information that includes the MAC address of the hardware in the end user device (or CPE)) is provided. The use of DHCP and CLIPS on the ND captures the MAC addresses and uses these addresses to distinguish subscribers and access their subscriber records.

[00121] A virtual circuit (VC), synonymous with virtual connection and virtual channel, is a connection-oriented communication service that is delivered by means of packet mode communication. Virtual circuit communication resembles circuit switching, since both are connection oriented, meaning that in both cases data is delivered in correct order, and signaling overhead is required during a connection establishment phase. Virtual circuits may exist at different layers. For example, at layer 4, a connection-oriented transport layer datalink protocol such as Transmission Control Protocol (TCP) may rely on a connectionless packet switching network layer protocol such as IP, where different packets may be routed over different paths, and thus be delivered out of order. Where a reliable virtual circuit is established with TCP on top of the underlying unreliable and connectionless IP protocol, the virtual circuit is identified by the source and destination network socket address pair, i.e., the sender and receiver IP address and port number. However, a virtual circuit is possible since TCP includes segment numbering and reordering on the receiver side to prevent out-of-order delivery. Virtual circuits are also possible at Layer 3 (network layer) and Layer 2 (datalink layer); such virtual circuit protocols are based on connection-oriented packet switching, meaning that data is always delivered along the same network path, i.e., through the same NEs/VNEs. In such protocols, the packets are not routed individually, and complete addressing information is not provided in the header of each data packet; only a small virtual channel identifier (VCI) is required in each packet; and routing information is transferred to the NEs/VNEs during the connection establishment phase; switching only involves looking up the virtual channel identifier in a table rather than analyzing a complete address. Examples of network layer and datalink layer virtual circuit protocols, where data always is delivered over the same path: X.25, where the VC is identified by a virtual channel identifier (VCI); Frame relay, where the VC is identified by a VCI; Asynchronous Transfer Mode (ATM), where the circuit is identified by a virtual path identifier (VPI) and virtual channel identifier (VCI) pair; General Packet Radio Service (GPRS); and Multiprotocol label switching (MPLS), which can be used for IP over virtual circuits (Each circuit is identified by a label).

[00122] Certain NDs (e.g., certain edge NDs) use a hierarchy of circuits. The leaf nodes of the hierarchy of circuits are subscriber circuits. The subscriber circuits have parent circuits in the hierarchy that typically represent aggregations of multiple subscriber circuits, and thus the network segments and elements used to provide access network connectivity of those end user devices to the ND. These parent circuits may represent physical or logical aggregations of subscriber circuits (e.g., a virtual local area network (VLAN), a permanent virtual circuit (PVC) (e.g., for Asynchronous Transfer Mode (ATM)), a circuit-group, a channel, a pseudowire, a physical NI of the ND, and a link aggregation group). A circuit-group is a virtual construct that allows various sets of circuits to be grouped together for configuration purposes, for example aggregate rate control. A pseudo-wire is an emulation of a layer 2 point-to-point connection-oriented service. A link aggregation group is a virtual construct that merges multiple physical NIs for purposes of bandwidth aggregation and redundancy. Thus, the parent circuits physically or logically encapsulate the subscriber circuits.

[00123] Each VNE (e.g., a virtual router, a virtual bridge (which may act as a virtual switch instance in a Virtual Private LAN Service (VPLS) is typically independently administrable. For example, in the case of multiple virtual routers, each of the virtual routers may share system resources but is separate from the other virtual routers regarding its management domain, AAA (authentication, authorization, and accounting) name space, IP address, and routing database(s). Multiple VNEs may be employed in an edge ND to provide direct network access and/or different classes of services for subscribers of service and/or content providers.

[00124] Within certain NDs, “interfaces” that are independent of physical NIs may be configured as part of the VNEs to provide higher-layer protocol and service information (e.g., Layer 3 addressing). The subscriber records in the AAA server identify, in addition to the other subscriber configuration requirements, to which context (e.g., which of the VNEs/NEs) the corresponding subscribers should be bound within the ND. As used herein, a binding forms an association between a physical entity (e.g., physical NI, channel) or a logical entity (e.g., circuit such as a subscriber circuit or logical circuit (a set of one or more subscriber circuits)) and a context’s interface over which network protocols (e.g., routing protocols, bridging protocols) are configured for that context. Subscriber data flows on the physical entity when some higher-layer protocol interface is configured and associated with that physical entity.

[00125] Some NDs provide support for implementing VPNs (Virtual Private Networks) (e.g., Layer 2 VPNs and/or Layer 3 VPNs). For example, the ND where a provider’s network and a customer’s network are coupled are respectively referred to as PEs (Provider Edge) and CEs (Customer Edge). In a Layer 2 VPN, forwarding typically is performed on the CE(s) on either end of the VPN and traffic is sent across the network (e.g., through one or more PEs coupled by other NDs). Layer 2 circuits are configured between the CEs and PEs (e.g., an Ethernet port, an ATM permanent virtual circuit (PVC), a Frame Relay PVC). In a Layer 3 VPN, routing typically is performed by the PEs. By way of example, an edge ND that supports multiple VNEs may be deployed as a PE; and a VNE may be configured with a VPN protocol, and thus that VNE is referred as a VPN VNE.

[00126] Some NDs provide support for VPLS (Virtual Private LAN Service). For example, in a VPLS network, end user devices access content/services provided through the VPLS network by coupling to CEs, which are coupled through PEs coupled by other NDs. VPLS networks can be used for implementing triple play network applications (e.g., data applications (e.g., high-speed Internet access), video applications (e.g., television service such as IPTV (Internet Protocol Television), VoD (Video-on-Demand) service), and voice applications (e.g., VoIP (Voice over Internet Protocol) service)), VPN services, etc. VPLS is a type of layer 2 VPN that can be used for multi-point connectivity. VPLS networks also allow end use devices that are coupled with CEs at separate geographical locations to communicate with each other across a Wide Area Network (WAN) as if they were directly attached to each other in a Local Area Network (LAN) (referred to as an emulated LAN).

[00127] In VPLS networks, each CE typically attaches, possibly through an access network (wired and/or wireless), to a bridge module of a PE via an attachment circuit (e.g., a virtual link or connection between the CE and the PE). The bridge module of the PE attaches to an emulated LAN through an emulated LAN interface. Each bridge module acts as a “Virtual Switch Instance” (VSI) by maintaining a forwarding table that maps MAC addresses to pseudowires and attachment circuits. PEs forward frames (received from CEs) to destinations (e.g., other CEs, other PEs) based on the MAC destination address field included in those frames.

[00128] Figure 9 illustrates an apparatus 900 including a processor 902, according to some embodiments. The apparatus, 900, may include a processing circuitry (one or more than one processors), 902, coupled to an interface, 908, and to the memory 904. The apparatus, 900, may comprise more than one interface. By way of example, the interface 908, the processor(s) 902, and the memory 904 may be connected in series as illustrated in Figure 9. Alternatively, these components 902, 904 and 908 may be coupled to an internal bus system of the apparatus, 900. The memory 904 may include a Read-Only-Memory (ROM), e.g., a flash ROM, a Random Access Memory (RAM), e g., a Dynamic RAM (DRAM) or Static RAM (SRAM), a mass storage, e.g., a hard disk or solid state disk, or the like. The memory, 904, may contain a computer program (software or instructions), 906, and/or control parameters. The memory, 904, may include suitably configured program code to be executed by the processor(s), 902, so as to implement the above-described method as explained in connection with Figures 1 - 6.

[00129] Figure 10 illustrates an exemplary Open Radio Access Network (O-RAN) compliant network 1000, according to some embodiments of the invention. As shown in Figure 10, O-RAN compliant network 1000 includes intent owner 1005 and service management and orchestration (SMO) component 1010. Intent owner 1005 is coupled to SMO component 1010 through intent interface (I/F) 1002 to network node 1015. In some embodiments, intent owner 1005 communicates intents to network node 1015 through intent interface 1002.

[00130] In some embodiments, as shown in Figure 10, O-RAN compliant network 1000 includes a near real-time RAN intelligent controller (RIC) 1020 coupled to one or more nodes that can operate an open interface between two end points (e.g., E2 interface). For example, near real-time RIC 1020 is coupled to O-RAN E2 nodes O-eNB 1022, CU-CP 1024, CU-UP 1026 and DU 1025. In some embodiments, O-RAN E2 nodes O-eNB 1022, CU-CP 1024, CU-UP 1026 and DU 1025 are implemented in a virtualization environment (described in further detail below) in which one or more network functions for O-RAN compliant network 1000 are virtualized. In one embodiment, the virtualization environment (e.g., O-RAN compliant network 1000) includes O-Cloud computing platform 1035. In some embodiments, O-Cloud computing platform 1035 is implemented by SMO component 1010 via an O-2 interface defined by the O-RAN Alliance or comparable technologies.

[00131] In some embodiments, near real-time RIC 1020 separates functions of the O-RAN compliant network 1000 into a centralized unit (e.g., CU-CP 1024 and CU-UP 1026) and a distributed unit (e.g., DU 1025 and RU 1030). In some embodiments, the physical layer for O-RAN compliant network 1000 is split with the lower part of the physical layer running in a radio unit (e.g., RU 1030) and the upper part of the physical layer running in a distributed unit (e.g., DU 1025). In some embodiments, as shown in Figure 10, the centralized unit (CU) is split into a centralized unit control plane (CU-CP 1024) and a centralized unit user place (CU- UP 1026). In some embodiments, RU 1030 is a physical appliance and O-eNb 1022, CU-UP 1024, CU-UP 1026, and/or DU 1025 are either physical appliances or virtualized instances running over an O-cloud layer (e.g., O-cloud 1035). In some embodiments, a next generation (NG) core is communicatively connected to CU-UP 1024, CU-CP 1026, DU 1025, and RU 1030.

[00132] In some embodiments, O-RAN compliant network 1000 includes interfaces 01, Al, 02, Xn (e.g., Xn-c and XN-u), and NG (e.g., NG-c and NG-u) interfaces as shown. Although intent interface 1002 is illustrated as communicating only with SMO component 1010, examples herein are not so limited. For example, in some embodiments, network node 1015 of SMO component 1010 is coupled with near real-time RIC 1020 via an intent interface (e.g., intent interface 1002). In some embodiments, network node 1015 is a part of near real-time RIC 1020. For example, network node 1015 is located inside near real-time RIC 1020 and coupled with CU-CP 1024, CU-UP 1026, DU 1025, and/or RU 1030 via an intent interface (e.g., intent interface 1002).

[00133] 0-RAN compliant network 1000 can act as intent tester system 125 and/or as the intent system under test 135. For example, in some embodiments, intent owner 1005 is implemented as intent owner 105 and send test environment information to network node 1015. In such an example, network node 1015, acting as the intent tester system (e.g., intent tester system 125 of Figure 1), generates intents under test to be sent to other network nodes within the O-RAN compliant network 1000. In other embodiments, the intent owner 1005 is implemented as an intent tester system (e.g., intent tester system 125 of Figure 1) and sends intents under test (e.g., intent under test 108 of Figure 1) to network node 1015. In such embodiments, network node 1015 would receive the intents under test, implement the intent test and send intent test report (e.g., intent test report 110 of Figure 1) to intent owner 1005. [00134] An embodiment may be an article of manufacture in which a non-transitory machine-readable storage medium (such as microelectronic memory) has stored thereon instructions (e.g., computer code) which program one or more data processing components (generically referred to here as a “processor”) to perform the operations described above. In other embodiments, some of these operations might be performed by specific hardware components that contain hardwired logic (e.g., dedicated digital filter blocks and state machines). Those operations might alternatively be performed by any combination of programmed data processing components and fixed hardwired circuit components.

[00135] While the invention has been described in terms of several embodiments, those skilled in the art will recognize that the invention is not limited to the embodiments described, can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative instead of limiting.

[00136] For example, while the flow diagrams in the figures show a particular order of operations performed by certain embodiments of the invention, it should be understood that such order is exemplary (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).

Claims

CLAIMS What is claimed is:

1. A method for an intent tester system implemented by a first electronic device, the method comprising: receiving (605), at the intent tester system implemented by the first electronic device, a request from an intent owner to implement an intent test, wherein the request comprises service requirements and test environment information; determining (610) a set of intent systems under test using the service requirements; generating (615) an intent under test for each of the set of intent systems using the service requirements and the test environment information; sending (620), to the set of intent systems under test, the intents under test; receiving (625), from the set of intent systems under test, intent test reports in response to sending the intents under test, wherein the intent test reports include information about the set of intent systems testing the intents under test; generating (630), using the intent test reports from the intent systems under test, a response to the request to implement the intent test; and sending (635), to the intent owner, the response.

2. The method of claim 1, wherein the intents under test include test requirements for the intent test and test conditions for the intent test.

3. The method of claim 2, wherein an intent test report of the intent test reports is generated in response to simulating the test conditions on a network model and testing the test requirements on the network model.

4. The method of claim 2, wherein an intent test report of the intent test reports is generated in response to testing the test requirements on a network of the intent system under test.

5. The method of claim 4, further comprising: determining, by the intent system under test, that current conditions of the network of the intent system under test satisfy the test conditions, wherein testing the test requirements is in response to the determination.

6. The method of any of claims 2-5, wherein the test environment information comprises rules and a scope of the intent test, the method further comprising: generating, using the rules, the scope of the intent test, and the service requirements, the test requirements and the test conditions.

7. The method of claim 6, wherein generating the test requirements and the test conditions comprises: applying a trained machine learning model to the rules, the scope of the intent test, and the service requirements; and outputting, by the trained machine learning model, the test requirements and the test conditions.

8. The method of any of claims 1-7, wherein generating the response to the request comprises: applying a trained machine learning model to the intent test reports, wherein the response to the request is generated using an output of the trained machine learning model.

9. The method of any of claims 1-8, further comprising: determining to terminate the intent test; and terminating the intent test in response to the determination.

10. The method of claim 9, wherein the request further comprises a termination condition and wherein determining to terminate the intent test is based on determining that the termination condition is met.

11. A machine-readable medium comprising computer program code which when executed by a computer carries out the method steps of any of claims 1-10.

12. A first electronic device implementing an intent tester system, the first electronic device comprising: a processor and a memory, the memory containing instructions executable by the processor whereby the first electronic device is operative to: receive (605), at the intent tester system implemented by the first electronic device, a request from an intent owner to implement an intent test, wherein the request comprises service requirements and test environment information; determine (610) a set of intent systems under test using the service requirements; generate (615) an intent under test for each of the set of intent systems using the service requirements and the test environment information; send (620), to the set of intent systems under test, the intents under test; receive (625), from the set of intent systems under test, intent test reports in response to sending the intents under test, wherein the intent test reports include information about the set of intent systems testing the intents under test; generate (630), using the intent test reports from each of the intent systems under test, a response to the request to implement the intent test; and send (635), to the intent owner, the response.

13. The first electronic device of claim 12, wherein the intents under test include test requirements for the intent test and test conditions for the intent test.

14. The first electronic device of claim 13, wherein an intent test report of the intent test reports is generated in response to simulating the test conditions on a network model and testing the test requirements on the network model.

15. The first electronic device of claim 14, wherein an intent test report of the intent test reports is generated in response to testing the test requirements on a network of the intent system under test.

16. The first electronic device of claim 15, wherein the first electronic device is further operative to: determine, by the intent system under test, that current conditions of the network of the intent system under test satisfy the test conditions, wherein testing the test requirements is in response to the determination.

17. The first electronic device of any of claims 13-16, wherein the test environment comprises rules and a scope of the intent test and wherein the first electronic device is further operative to: generate, using the rules, the scope of the intent test, and the service requirements, the test requirements and the test conditions.

18. The first electronic device of claim 17, wherein generating the test requirements and the test conditions comprises: applying a trained machine learning model to the rules, the scope of the intent test, and the service requirements; and outputting, by the trained machine learning model, the test requirements and the test conditions.

19. The first electronic device of any of claims 12-18, wherein generating the response to the request comprises: applying a trained machine learning model to the intent test reports, wherein the response to the request is generated using an output of the trained machine learning model.

20. The first electronic device of any of claims 12-19, wherein the first electronic device is further operative to: determine to terminate the intent test; and terminate the intent test in response to the determination.

21. The first electronic device of claim 20, wherein the request further comprises a termination condition and wherein determining to terminate the intent test is based on determining that the termination condition is met.