CN117917877A

CN117917877A - Dialogue assistant for site troubleshooting

Info

Publication number: CN117917877A
Application number: CN202311104980.5A
Authority: CN
Inventors: 吴小英; K·沙; M·P·内吉
Original assignee: Juniper Networks Inc
Current assignee: Juniper Networks Inc
Priority date: 2022-10-20
Filing date: 2023-08-30
Publication date: 2024-04-23

Abstract

Embodiments of the present disclosure relate to a conversation assistant for site troubleshooting. A Network Management System (NMS) is described that includes one or more processors coupled to a memory storing network data. The one or more processors are configured to receive a query identifying a site, and determine a first set of troubleshooting problems for Wide Area Network (WAN) deployment at the site, a second set of troubleshooting problems for wireless deployment at the site, and a third set of troubleshooting problems for wired deployment at the site based on the network data. The one or more processors are configured to determine, based on the user experience metrics, a first troubleshooting problem from a first set of troubleshooting problems for the WAN deployment, a second troubleshooting problem from a second set of troubleshooting problems for the wireless deployment, and a third troubleshooting problem from a third set of troubleshooting problems for the wired deployment.

Description

Dialogue assistant for site troubleshooting

The present application claims the benefit of U.S. patent application Ser. No. 18/343,914, filed on Ser. No. 29 at 6 at 2023, which claims the benefit of U.S. provisional patent application Ser. No. 63/380,314, filed on Ser. No. 20 at 10 at 2022, which is incorporated herein by reference in its entirety.

Technical Field

The present disclosure relates generally to computer networks and, more particularly, to monitoring and troubleshooting computer networks.

Background

A business or site, such as an office, hospital, airport, stadium, or retail store, typically installs a complex wireless network system throughout the site, including a wireless Access Point (AP) network, to provide wireless network services to one or more wireless client devices (or simply "clients"). An AP is a physical, electronic device that enables other devices to wirelessly connect to a wired network using various wireless networking protocols and technologies, such as a wireless local area network networking protocol (i.e., "Wi-Fi"), bluetooth/Bluetooth Low Energy (BLE), a mesh networking protocol such as ZigBee, or other wireless networking technologies that conform to one or more of the IEEE 802.11 standards. Many different types of wireless client devices, such as notebook computers, smartphones, tablet computers, wearable devices, appliances, and internet of things (IoT) devices, employ wireless communication technology and may be configured to connect to a wireless access point when the device is within range of a compatible wireless access point in order to access a wired network. In the case of a client device running a cloud-based application, such as a Voice Over Internet Protocol (VOIP) application, streaming video application, gaming application, or video conferencing application, data is exchanged from the client device to reach a cloud-based application server during an application session through one or more APs and one or more wired network devices, e.g., switches, routers, and/or gateway devices.

Disclosure of Invention

In general, this disclosure describes one or more techniques for a Network Management System (NMS) to identify network problems that occur within a particular network site and to provide a summary that indicates what, if any, problems have occurred for each of the Wide Area Network (WAN), wired, and wireless deployments for that particular network site. For each type of deployment (e.g., wireless, wired, WAN), the summary may indicate a corresponding set of questions (e.g., one or more questions). For example, for wireless deployments, problems may include authentication failure, poor coverage, and/or disconnection of an Access Point (AP). For each deployment type for a given site, the NMS may classify each problem into a troubleshooting category (e.g., a client category, a connectivity category, and a device health category). For example, the NMS may classify each problem as one of a client class, connectivity class, or device health class for wireless deployment. For each troubleshooting category, the NMS may rank the primary questions of each deployment using a user experience metric such as a Service Level Expectancy (SLE) score as an indicator (e.g., primary indicator). In some examples, the NMS can rank the first questions of each deployment (e.g., the first questions of each of wireless, wired, and WAN) based on SLE scores. The NMS may display only the first questions in each troubleshooting category or a set number of first questions (e.g., 2 questions, 3 questions, etc.) in each troubleshooting category. Additionally or alternatively, the NMS may rank individual primary questions (e.g., primary questions for each of wireless, wired, and WAN) in all deployments based on SLE scores.

In accordance with the techniques of this disclosure, an NMS may identify problems experienced by client devices on a site based on network data or simply "data" collected from network devices within the wireless deployment of the site. For example, the NMS may determine that a client device on a site experiences an ethernet error problem based on data collected from Access Point (AP) devices within the wireless deployment of the site. As another example, the NMS may identify problems experienced by switches on the site based on data collected from network devices within the wired deployment of the site. For example, the NMS may determine from data collected from switches and routers within a wired deployment of a site that a switch on the site experienced a switch disconnection problem. The NMS may cause a primary problem of outputting each deployment. For example, in response to determining that the ethernet error has the highest SLE score for the wireless deployment, the NMS may output a visual indication of the ethernet error in the conversation assistant. Similarly, in response to determining that the switch disconnect problem has the highest SLE score for the wired deployment, the NMS may output a visual indication in the dialogue assistant of the switch disconnect problem.

The disclosed technology is capable of troubleshooting sites by identifying network problems at each deployment. For example, the NMS may access not only wireless data to determine problems at the site, but may also further access wired data and/or WAN data. In this way, the NMS may determine wired problems and/or WAN problems, which may help reduce the time an administrator spends on site troubleshooting. Furthermore, the NMS may troubleshoot WAN, wireless, and wired deployments simultaneously, and may determine root causes for all deployment types faster than systems that use data from wireless deployments alone to determine root causes. Furthermore, the NMS may suggest performing some action in the dialogue assistance to quickly solve the problem(s).

The details of one or more examples of the technology of the present disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the technology will be apparent from the description and drawings, and from the claims.

Drawings

FIG. 1A is a block diagram of an example network system in which a network management system provides troubleshooting for sites in accordance with one or more techniques of the present disclosure.

Fig. 1B is a block diagram illustrating further example details of the network system of fig. 1A.

Fig. 2 is a block diagram of an example access point device in accordance with one or more techniques of this disclosure.

Fig. 3 is a block diagram of an example network management system configured to provide failover for a site in accordance with one or more techniques of this disclosure.

Fig. 4 is a block diagram of an example user equipment device in accordance with one or more techniques of this disclosure.

Fig. 5 is a block diagram of an example network node, such as a router or switch, in accordance with one or more techniques of this disclosure.

FIG. 6 illustrates an example user interface of a network management system for visualizing troubleshooting a site in accordance with one or more techniques of the present disclosure.

FIG. 7 is a flowchart illustrating example operations of site troubleshooting in accordance with one or more techniques of the present disclosure.

Detailed Description

Fig. 1A is a block diagram of an example network system 100 in which a Network Management System (NMS) 130 provides troubleshooting for sites in accordance with one or more techniques of the present disclosure. The example network system 100 includes a plurality of sites 102A-102N at which network service providers manage one or more wireless networks 106A-106N, respectively. Although in fig. 1A, each station 102A-102N is shown as including a single wireless network 106A-106N, respectively, in some examples, each station 102A-102N may include multiple wireless networks, and the disclosure is not limited thereto.

Each site 102A-102N includes a plurality of Network Access Server (NAS) devices, such as Access Points (APs) 142, switches 146, or routers (not shown) within the edge of a wired network. For example, station 102A includes a plurality of APs 142A-1 through 142A-M. Similarly, station 102N includes a plurality of APs 142N-1 through 142N-M. Each AP 142 may be any type of wireless access point including, but not limited to, a business or enterprise AP, a router, or any other device connected to a wired network and capable of providing wireless network access to client devices within a site.

Each station 102A-102N also includes a plurality of client devices, also referred to as user equipment devices (UEs), collectively referred to as UEs or client devices 148, representing the various wireless-enabled devices within each station. For example, a plurality of UEs 148A-1 through 148A-N are currently located at site 102A. Similarly, a plurality of UEs 148N-1 through 148N-N are currently located at site 102N. Each UE 148 may be any type of wireless client device including, but not limited to, a mobile device such as a smart phone, tablet or laptop computer, personal Digital Assistant (PDA), wireless terminal, smart watch, smart ring, or other wearable device. UE 148 may also include a wired client device, e.g., an IoT device such as a printer, security device, environmental sensor, or any other device connected to a wired network and configured to communicate over one or more wireless networks 106.

To provide wireless network services to UEs 148 and/or communicate over wireless network 106, AP 142 and other wired client devices at site 102 are directly or indirectly connected to one or more network devices (e.g., switches, routers, etc.) via physical cables (e.g., ethernet cables). In the example of fig. 1A, station 102A includes a switch 146A, and each of the APs 142A-1 through 142A-M at station 102A is connected to the switch 146A. Similarly, station 102N includes a switch 146N to which each of the APs 142N-1 through 142N-M at station 102N are connected. Although shown in fig. 1A as if each station 102 included a single switch 146 and all APs 142 of a given station 102 were connected to a single switch 146, in other examples, each station 102 may include more or fewer switches and/or routers. Further, APs and other wired client devices of a given site may be connected to two or more switches and/or routers. Furthermore, two or more switches at a site may be connected to each other and/or to two or more routers, e.g., via a mesh or partial mesh topology in a hub-and-spoke architecture. In some examples, the interconnected switches and routers include a wired Local Area Network (LAN) at site 102 hosting wireless network 106.

Example network system 100 also includes various networking components for providing networking services within a wired network, including, for example, authentication, authorization, and accounting (AAA) server 110 for authenticating users and/or UEs 148, dynamic Host Configuration Protocol (DHCP) server 116 for dynamically assigning network addresses (e.g., IP addresses) to UEs 148 upon authentication, domain Name System (DNS) server 122 for resolving domain names to network addresses, multiple servers 128A-128X (collectively, "servers 128") (e.g., network servers, database servers, file servers, etc.), and Network Management System (NMS) 130. As shown in fig. 1A, various devices and systems of network 100 are coupled together via one or more networks 134 (e.g., the internet and/or an enterprise intranet).

In the example of fig. 1A, NMS130 is a cloud-based computing platform that manages wireless networks 106A through 106N at one or more of sites 102A through 102N. As further described herein, NMS130 provides an integrated management tool suite and implements the various techniques of the disclosure. In general, NMS130 may provide a cloud-based platform for network data acquisition, monitoring, activity logging, reporting, predictive analysis, network anomaly identification, and alarm generation. In some examples, NMS130 outputs notifications, such as alarms, graphical indicators on the dashboard, log messages, text/SMS messages, email messages, etc., and/or suggestions regarding network problems, to a site or network administrator ("administrator") interacting with administrator device 111 and/or operating administrator device 111. Further, in some examples, NMS130 operates in response to configuration inputs received from an administrator interacting with administrator device 111 and/or operating administrator device 111.

The administrator and administrator device 111 may include IT personnel and administrator computing devices associated with one or more sites 102 and/or switches 146 at the edge of the wired network. The administrator device 111 may be implemented as any suitable device for presenting output and/or accepting user input. For example, the administrator device 111 may include a display. The administrator device 111 may be a computing system, such as a mobile or non-mobile computing device operated by a user and/or by an administrator. In accordance with one or more aspects of the present disclosure, administrator device 111 may represent, for example, a workstation, a laptop or notebook computer, a desktop computer, a tablet computer, or any other computing device operable by a user and/or presenting a user interface. The administrator device 111 may be physically separate from the NMS130 and/or located in a different location from the NMS130 such that the administrator device 111 may communicate with the NMS130 via the network 134 or other communication means.

In some examples, one or more of the NAS devices (e.g., AP 142, switch 146, or router) may be connected to edge devices 150A-150N via a physical cable (e.g., ethernet cable). Edge device 150 includes a cloud managed wireless Local Area Network (LAN) controller. Each edge device 150 may comprise a local device at site 102 that communicates with NMS130 to extend some micro services from NMS130 to local NAS devices while using NMS130 and its distributed software architecture for scalable and resilient operation, management, troubleshooting, and analysis.

Each of the network devices of network system 100, such as servers 110, 116, 122 and/or 128, AP 142, UE 148, switch 146, and any other servers or devices attached to or forming part of network system 100, may include a system log or error log module, where each of these network devices records the status of the network device including normal operating status and error conditions. Throughout this disclosure, one or more of the network devices of network system 100, such as servers 110, 116, 122 and/or 128, AP 142, UE 148, and switch 146, may be considered "third party" network devices when owned by and/or associated with an entity other than NMS130, such that NMS130 does not receive, collect, or otherwise access the recorded status and other data of the third party network devices. In some examples, edge device 150 may provide an agent through which the status and other data of the logged third party network device may be reported to NMS 130.

In some examples, NMS130 monitors network data 137, such as one or more Service Level Expectation (SLE) metrics, received from wireless networks 106A through 106N at each site 102A through 102N, respectively, and manages network resources, such as AP 142 at each site, to provide high quality wireless experiences to end users, ioT devices, and clients at the site. For example, NMS130 may include a Virtual Network Assistant (VNA) 133 that implements an event handling platform for providing real-time insight and simplified troubleshooting for IT operations, and automatically taking corrective actions or providing advice to proactively solve network problems. For example, VNA 133 may include an event processing platform configured to process hundreds or thousands of concurrent network data streams 137 from sensors and/or agents associated with APs 142 and/or nodes within network 134. For example, VNA 133 of NMS130 may include a base analysis and network error identification engine and an alarm system according to various examples described herein. The underlying analysis engine of the VNA 133 may apply historical data and models to the inbound event stream to calculate an assertion, such as an identified anomaly or predicted occurrence of an event that constitutes a network error condition. In addition, VNA 133 may provide real-time alarms and reports to notify a site or network administrator of any predicted events, anomalies, trends via administrator device 111, and may perform root cause analysis and automatic or assisted error remediation. In some examples, VNA 133 of NMS130 may apply machine learning techniques to identify the root cause of the error condition detected or predicted from network data stream 137. If the root cause can be automatically resolved, the VNA 133 can invoke one or more corrective actions to correct the root cause of the error condition, thereby automatically improving the underlying SLE metrics and also automatically improving the user experience.

Further example details of the operations implemented by VNA 133 of NMS130 are described below: U.S. patent 9,832,082, issued on 28/11/2017, entitled "Monitoring WIRELESS ACCESS Point Events" (Monitoring wireless access Point Events); U.S. publication No. US2021/0306201, published at 9/30 of 2021, entitled "Network System Fault Resolution Using A MACHINE LEARNING Model" (network system failure solution using machine learning Model); U.S. patent 10,985,969, issued at 2021, month 4, and 20, and entitled "SYSTEMS AND Methods for a Virtual Network Assistant" (systems and methods of virtual network assistant); U.S. patent 10,958,585, issued 2021, 3, 23, and entitled "Methods and Apparatus for Facilitating Fault Detection and/or Predictive Fault Detection" (methods and apparatus for facilitating fault detection and/or predictive fault detection); U.S. patent 10,958,537, issued 2021, 3/23, and entitled "Method for Spatio-Temporal Modeling" (space-time modeling method); and U.S. patent 10,862,742, issued at 2021, 12/8 and entitled "Method for Conveying AP Error Codes Over BLE Advertisements" (method for transmitting AP error codes on BLE announcements), the entire contents of which are incorporated herein by reference.

In operation, NMS130 may observe, collect and/or receive network data 137, which network data 137 may take the form of data extracted from messages, counters and statistics, for example. The network data 137 may include one or more of wireless data for a wireless deployment (e.g., wireless network 106A), wired data for a wired deployment (e.g., switch 146A), or a WAN deployment.

According to one specific implementation, the computing device is part of NMS 130. According to other implementations, NMS130 may include one or more computing devices, dedicated servers, virtual machines, containers, services, or other forms of environments for performing the techniques described herein. Similarly, the computing resources and components implementing VNA 133 may be part of NMS130, may execute on other servers or execution environments, or may be distributed to nodes (e.g., routers, switches, controllers, gateways, etc.) within network 134.

Some solutions to identify network problems that occur within a particular network site include identifying network problems related to client, connectivity, and device health of a single deployment type (e.g., wireless deployment) of the particular network site. This may result in proactive identification and potential avoidance of problems before a single deployment type is problematic and/or reduced problem solving time when a single deployment type is truly problematic. However, such solutions do not identify problems across multiple deployment types in a similar manner. For example, problems occurring in a wired or WAN deployment of a site may also cause problems in a wireless deployment of the site, which may be difficult or even undetectable using only wireless data collected from network devices in the wireless deployment. Thus, an administrator or other user may need to manually identify questions across multiple deployment types, for example, by making separate queries to different troubleshooting engines for each deployment type in a site.

For example, the wireless troubleshooting engine may troubleshoot wireless problems, but may not be able to provide a useful root cause of the problem based solely on wireless data. As described above, problems in wireless deployments may be caused by wired or WAN problems. By troubleshooting the WAN, wireless, and wired together, the site troubleshooting engine 135 can identify root causes for all deployment types. In addition, the site troubleshooting engine 135 may recommend actions to quickly solve the problem(s).

The techniques described herein may include the site troubleshooting engine 135 identifying network problems occurring within a particular network site and generating a summary indicating what the problems are for each of the WAN, wired, and wireless deployments of the particular network site. For each type of deployment, the summary may indicate a corresponding set of questions. For example, for wireless deployments, a set of one or more problems may include inability to connect and slower connection speeds. The problem of inability to connect in a wireless deployment may include one or more of authentication failure or disconnection of an access point. Problems with connections but slower speeds in wireless deployments may include one or more of poor access point coverage or lower capacity. Similarly, for wired deployments, a set of one or more problems may include inability to connect and slower connection speeds. The problem of inability to connect in a wired deployment may include one or more of authentication failure or disconnection of an access point. Problems with connections but slower speeds in wired deployments may include one or more of poor coverage or low capacity of the access point.

For each deployment type (e.g., wireless, wired, and/or WAN deployment) for a given site, the site troubleshooting engine 135 can classify each problem as client, connectivity, and device health (i.e., three troubleshooting categories). For each troubleshooting category, the site troubleshooting engine 135 can rank the primary questions using the SLE score as the primary indicator. As discussed further below, the site troubleshooting engine 135 may rank the questions and/or rank the primary questions in the deployment across the troubleshooting categories of each deployment. As shown in table 1, the site troubleshooting engine 135 may report only the primary problem in each category. In some examples, the conversation assistant engine 136 can display table 1 in a conversation assistant and/or a dashboard of the conversation assistant.

	WAN	Wired wire	Wireless communication system
				Client terminal	Internet connectivity loss	Slow performance	Authentication failure
Connectivity	Authentication failure	Slow performance	Poor coverage
				Device health	Disconnecting the connection	Slow performance	AP disconnect

TABLE 1 site A problem

The site troubleshooting engine 135 may provide the user with the ability to drill down through the primary questions of each category to provide more details of each fault indicated in the summary (e.g., cause of the fault, percentage of affected clients, scope of impact). In some examples, site troubleshooting engine 135 can include only one redirect link to the corresponding SLE of that class. If the site has a WAN deployment, the site troubleshooting engine 135 may include application categories as shown in Table 2. The application category may identify a cause of the application problem. In some examples, the conversation assistant engine 136 can display table 2 in a conversation assistant and/or a dashboard of the conversation assistant.

	WAN	Wired wire	Wireless communication system
				Client terminal	Internet connectivity loss	Slow performance	Authentication failure
Connectivity	Authentication failure	Slow performance	Poor coverage
				Device health	Disconnecting the connection	Slow performance	AP disconnect
Application of	Problem of application	-	-

TABLE 2 site B problem

The site troubleshooting engine 135 can determine that the application problem is the primary problem for WAN deployment. For example, the site troubleshooting engine 135 may be preconfigured to rank application problems in the application category higher than all problems in the client, connectivity, and device health categories. In this example, the site troubleshooting engine 135 may rank slow performance in the connectivity category of the wired deployment as a primary issue for the wired deployment. For example, the site troubleshooting engine 135 may be preconfigured to rank the problems in the connectivity category higher than all the problems in the client and device health categories for WAN and wireless deployments in response to identifying the application problems in the WAN deployment. In some instances, the site troubleshooting engine 135 can be configured to rank the slow performance problems in the connectivity category of the wired deployment higher than the problems in the client and device health categories in response to determining that the slow performance problems in the connectivity category have the highest SLE score.

In this example, the dialog assistant engine 136 may display application questions for WAN deployments, slow performance in connectivity categories for wired deployments, and slow performance in connectivity categories for wireless deployments. In this example, the conversation assistant engine 136 can avoid displaying problems in one or more of the client class or the device health class. For example, the conversation assistant engine 136 can avoid the problem of the display site troubleshooting engine 135 not identifying it as a primary problem for the deployment.

After the troubleshooting category, the dialog assistant engine 136 can display text indicating whether there are any pending actions for the site. Examples of pending actions may include suggested actions that have been determined but have not yet been performed. If there are any pending actions for the site, the dialog assistant engine 136 can indicate how many actions are pending for the site, and a redirection link to the site view of the action and the newly recommended action. Table 3 shows examples of actions that may be displayed.

TABLE 3 site C problem

For example, the site troubleshooting engine 135 may perform root cause analysis for authentication failures in the client class of the wireless deployment. In this example, the site troubleshooting engine 135 may identify pending action C. The conversation assistant engine 136 can display text indicating whether there is a pending action C for site C, for example, in the conversation assistant.

In some examples, a site or network administrator using administrator device 111, for example, may initiate troubleshooting of a particular site via conversation assistant engine 136 of VNA 133. The dialog assistant engine 136 may be configured to process user input, such as text strings, and generate responses. In some examples, the dialog assistant engine 136 may include one or more natural language processors configured to process user input. The conversation assistant engine 136 may be configured to conduct chat conversations that simulate the manner in which humans act as conversation partners, which may help simplify and/or improve administrator satisfaction with monitoring and controlling the network.

In accordance with one or more techniques of this disclosure, the conversation assistant engine 136 can generate a conversation assistant configured to receive user input. In a particular use case, an administrator may input a query to a particular site into conversation assistant engine 136 via administrator device 111. The dialog assistant engine 136 may provide a platform in which to present troubleshooting problems to an administrator via the administrator device 111.

After identifying a particular site, the site troubleshooting engine 135 may identify troubleshooting problems at various deployments at the site based on data collected from network devices within the various deployments, which may be retrieved from the time graph database 138. The site troubleshooting engine 135 may generate data representing the troubleshooting problems for presentation to an administrator within a conversation assistant using the administrator device 111. The visualization includes color coding, icons, or other indicia of the troubleshooting problems at the site deployment determined by the site troubleshooting engine 135 based on time data stored as network data 137 and/or time graph database 138. In this example, the administrator using the administrator device 111 may interact with the troubleshooting problems presented within the conversation assistant to select the troubleshooting problems for presentation to the site in response to the troubleshooting engine 135, or may additionally recommend to the user interface in connection with the user interface; submitted at day 1 and 13 of 2022, U.S. patent application Ser. No. 17/647,954 (docket number: JNP3538-US/2014-515US 01), entitled "CONVERSATIONAL ASSISTANT FOR OBTAINING NETWORK INFORMATION" (dialog assistant for obtaining network information), the entire contents of which are incorporated herein by reference.

The techniques of the present invention provide one or more technical advantages and practical applications. For example, these techniques enable site troubleshooting by identifying network problems in each deployment. For example, the site troubleshooting engine 135 may access not only wireless data to determine problems at the site, but also wired data and/or WAN data. In this way, the site troubleshooting engine 135 may determine wired issues and/or WAN issues, which may help reduce the amount of time an administrator spends on site troubleshooting. Further, the site troubleshooting engine 135, along with the WAN, wireless, and wired deployments, can identify root causes for all deployment types faster than systems that use only data from wireless deployments to identify root causes. In addition, the site troubleshooting engine 135 can recommend actions in the conversation assistance to quickly solve the problem(s).

Although the techniques of this disclosure are described in this example as being performed by NMS130, the techniques described herein may be performed by any other computing device(s), system(s), and/or server(s), and the disclosure is not limited thereto. For example, one or more computing devices configured to perform the functions of the techniques of this disclosure may reside in a dedicated server or may be included in any other server than NMS130 or may be distributed throughout network 100 and may or may not form part of NMS 130.

Fig. 1B is a block diagram illustrating further example details of the network system of fig. 1A. In this example, fig. 1B shows NMS130 configured to operate in accordance with an artificial intelligence and/or machine learning-based computing platform that provides comprehensive automation, insight, and assurance (Wi-Fi assurance, wired assurance, and WAN assurance).

As described herein, NMS130 may provide an integrated set of management tools and implement the various techniques of the present disclosure. In general, NMS130 may provide a cloud-based platform for wireless network data acquisition, monitoring, activity logging, reporting, predictive analysis, network anomaly identification, and alarm generation. For example, the network management system 130 may be configured to actively monitor and adaptively configure the network 100 to provide autopilot capability. Further, the VNA 133 may include a natural language processing engine to provide AI-driven support and troubleshooting, anomaly detection, AI-driven location services, and AI-driven Radio Frequency (RF) optimization with reinforcement learning.

As shown in the example of fig. 1B, AI-driven NMS130 may provide configuration management, monitoring, and automatic supervision of a software-defined wide area network (SD-WAN) 177, which software-defined wide area network 177 serves as an intermediary network communicatively coupling wireless network 106 and wired local area network 175 to data center 179 and application services (e.g., multi-cloud application) 181. In general, the SD-WAN 177 provides seamless, secure, traffic engineering connectivity between the edge wired network 175 hosting the wireless network 106 (e.g., a branch or campus network) to "spoke" routers 187A of the edge wired network 175 further down the cloud stack toward the "center" router 187B of the cloud-based application service 181. SD-WAN 177 typically operates and manages an overlay network over an underlying physical Wide Area Network (WAN) that provides connectivity to geographically separated customer networks. In other words, SD-WAN 177 may extend Software Defined Networking (SDN) capabilities to WANs and allow network(s) to separate underlying physical network infrastructure from virtual network infrastructure and applications so that the network may be configured and managed in a flexible and extensible manner.

In some examples, the underlying router of SD-WAN 177 may implement a stateful, session-based routing scheme in which routers 187A, 187B dynamically modify the content of the original packet header sourced by client device 148 to direct traffic along a selected path (e.g., path 189) to application service 181 without the use of tunnels and/or additional labels. In this way, the routers 187A, 187B may be more efficient and scalable for large networks, as using tunnel-less, session-based routing may enable the routers 187A, 187B to obtain substantial network resources by eliminating the need to perform encapsulation and decapsulation at tunnel endpoints. Further, in some examples, each router 187A, 187B may independently perform path selection and traffic engineering to control packet flows associated with each session without using a centralized SDN controller for path selection and label distribution. In some examples, routers 187A, 187B implement session-based routing as Secure Vector Routing (SVR) provided by the prospective blogging network company.

Additional information about session-based routing and SVR is described below: U.S. patent No. 9,729,439, entitled "COMPUTER NETWORK PACKET FLOW CONTROLLER" (computer network packet flow controller), issued 8/2017; U.S. patent 9,729,682, entitled "NETWORK DEVICE AND METHOD FOR PROCESSING A SESSION USING A PACKET SIGNATURE" (NETWORK device and method for handling sessions using packet signatures), issued 8 months 8, 2017; U.S. patent No. 9,762,485, entitled "NETWORK PACKET FLOW CONTROLLER WITH EXTENDED SESSION MANAGEMENT" (network packet flow controller with extended session management), issued on month 9 and 12 2017; U.S. patent 9,871,748, entitled "ROUTER WITH OPTIMIZED STATISTICAL FUNCTIONALITY" (router with optimized statistics), issued on 1 month 16 2018; U.S. patent 9,985,883, entitled "NAME-BASED ROUTING SYSTEM AND METHOD" (NAME-BASED ROUTING system and METHOD), issued on 5.29 of 2018; U.S. patent No.10,200,264, entitled "LINK STATUS MONITORING BASED ON PACKET LOSSDETECTION" (link state monitoring based on packet loss detection), issued on month 2 and 5 of 2019; U.S. patent 10,277,506, entitled "STATEFUL LOAD BALANCING IN A STATELESS NETWORK" (stateful load balancing in stateless NETWORKs), issued on month 4 and 30 of 2019; U.S. patent No.10,432,522, entitled "NETWORK PACKET FLOW CONTROLLER WITH EXTENDED SESSION MANAGEMENT" (network packet flow controller with extended session management), issued on 10 months 1 in 2019; and U.S. patent number 11,075,824, entitled "IN-LINE PERFORMANCE MONITORING" (on-line performance monitoring), issued at 2021, 7/27, each of which is incorporated herein by reference IN its entirety.

In some examples, AI-driven NMS130 may enable intent-based configuration and management of network system 100, including enabling construction, presentation, and execution of intent-driven workflows for configuring and managing devices associated with wireless network 106, wired LAN network 175, and/or SD-WAN 177. For example, declarative requirements represent the required network component configuration without specifying the exact native device configuration and control flow. By utilizing declarative requirements, it is possible to specify what should be done, rather than how it should be done. Declarative requirements may be contrasted with imperative instructions describing the exact device configuration syntax and control flow that implements the configuration. By utilizing declarative requirements rather than imperative instructions, the burden on the user and/or user system to determine the exact device configuration needed to achieve the desired results for the user/system is reduced. For example, when using a variety of different types of devices from different suppliers, it is often difficult and burdensome to specify and manage accurate command instructions to configure each device of the network. The device type and kind of the network may dynamically change as new devices are added and device failures occur. Using different configuration protocols, syntax and software versions to manage the various different types of devices from different vendors to configure a network of cohesive devices is often difficult to implement. Thus, by requiring only the user/system to specify declarative requirements that specify desired results applicable to a variety of different types of devices, management and configuration of network devices becomes more efficient. Further example details and techniques of the intent-based network management system are described below: U.S. patent 10,756,983, entitled "Intent-Based analysis" (Intent Based analysis), and U.S. patent 10,992,543, entitled "Automation GENERATING AN INTENT-Based network model of an existing computer network" (automatic generation of an Intent Based network model of existing computer networks), each of which is incorporated herein by reference.

In accordance with the techniques described in this disclosure, for a particular application session, the site troubleshooting engine 135 of the VNA 133 may receive a query identifying a site (e.g., site 102A) of the plurality of sites (e.g., the plurality of sites 102A-102N). For example, a human administrator may use administrator device 111 to enter queries. In this example, the site troubleshooting engine 135 of the VNA 133 may determine a first set of troubleshooting problems for the SD-WAN 177 at the site, a second set of troubleshooting problems for the one or more wireless LANs 106 at the site, and a third set of troubleshooting problems for the wired LAN 175 at the site based on data received from the network devices of the network system 100 and stored at the network data 137. For example, the site troubleshooting engine 135 of the VNA 133 may determine one or more of a client problem, connectivity problem, or device health problem for the WAN deployment. Similarly, the site troubleshooting engine 135 of the VNA 133 may determine one or more of one or more client problems, connectivity problems, or device health problems for wireless and/or wired deployments.

The site troubleshooting engine 135 of the VNA 133 may determine a first troubleshooting problem from the first set of troubleshooting problems for the WAN deployment, a second troubleshooting problem from the second set of troubleshooting problems for the wireless deployment, and a third troubleshooting problem from the third set of troubleshooting problems for the wired deployment based on the user experience metrics. For example, the site troubleshooting engine 135 of the VNA 133 may determine that the first troubleshooting problem has the highest impact (e.g., highest SLE score) on the user experience of the first set of troubleshooting problems for the WAN deployment based on the user experience metrics (e.g., SLE score values). Similarly, the site troubleshooting engine 135 of the VNA 133 may determine that the second troubleshooting problem has the highest impact on the user experience of the second set of troubleshooting problems for the wireless deployment based on the user experience metrics and/or that the third troubleshooting problem has the highest impact on the user experience of the third set of troubleshooting problems for the wired deployment based on the user experience metrics.

The dialog assistant engine 136 may generate data representing a user interface for presentation on the administrator device 111 that includes a visualization of at least the first troubleshooting problem, the second troubleshooting problem, and the third troubleshooting problem. For example, the dialog assistant engine 136 may generate data representing the user interface shown in fig. 6.

The disclosed technology is capable of troubleshooting sites by identifying network problems at each deployment. For example, the site troubleshooting engine 135 of the VNA 133 may access not only wireless data to determine problems at the site, but also wired data and/or WAN data. In this way, the site troubleshooting engine 135 of the VNA 133 may determine wired and/or WAN problems, which may help reduce the amount of time an administrator spends on site troubleshooting. Furthermore, the simultaneous troubleshooting of WAN, wireless, and wired deployments may help the site troubleshooting engine 135 of the VNA 133 to identify root causes for all deployment types faster than systems that use only data from wireless deployments to identify root causes. In addition, the site troubleshooting engine 135 of the VNA 133 may recommend actions in the dialogue assistance to quickly solve the problem(s).

Fig. 2 is a block diagram of an example Access Point (AP) device 200 configured in accordance with one or more techniques of this disclosure. The example access point 200 shown in fig. 2 may be used to implement any AP 142 as shown and described herein with reference to fig. 1A. The access point 200 may include, for example, a Wi-Fi, bluetooth, and/or Bluetooth Low Energy (BLE) base station, or any other type of wireless access point.

In the example of fig. 2, access point 200 includes a wired interface 230, wireless interfaces 220A-220B, one or more processors 206, memory 212, and input/output 210, coupled together via bus 214 over which various elements may exchange data and information. The wired interface 230 represents a physical network interface and includes a receiver 232 and a transmitter 234 for transmitting and receiving network communications (e.g., packets). The wired interface 230 directly or indirectly couples the access point 200 to a wired network device within a wired network, such as one of the switches 146 of fig. 1A, via a cable, such as an ethernet cable.

First and second wireless interfaces 220A and 220B represent wireless network interfaces, respectively, and include receivers 222A and 222B, respectively, each including a receive antenna via which access point 200 may receive wireless signals from a wireless communication device, such as UE 148 of fig. 1A. The first and second wireless interfaces 220A and 220B also include transmitters 224A and 224B, respectively, each including a transmit antenna through which the access point 200 may transmit wireless signals to a wireless communication device, such as UE 148 of fig. 1A. In some examples, the first wireless interface 220A may include a Wi-Fi 802.11 interface (e.g., 2.4GHz and/or 5 GHz), and the second wireless interface 220B may include a bluetooth interface and/or a Bluetooth Low Energy (BLE) interface.

The processor(s) 206 are programmable hardware-based processors configured to execute software instructions, such as software instructions for defining software or a computer program, stored to a computer-readable storage medium (e.g., memory 212), such as a non-transitory computer-readable medium including a storage device (e.g., disk drive or optical drive) or memory (e.g., flash memory or RAM) or any other type of volatile or non-volatile memory, that stores instructions to cause the one or more processors 206 to perform the techniques described herein.

Memory 212 includes one or more devices configured to store programming modules and/or data associated with the operation of access point 200. For example, the memory 212 may include a computer-readable storage medium, such as a non-transitory computer-readable medium including a storage device (e.g., a disk drive or optical drive) or memory (such as flash memory or random access memory) or any other type of volatile or non-volatile memory, that stores instructions to cause the one or more processors 206 to perform the techniques described herein.

In this example, memory 212 stores executable software including an Application Programming Interface (API) 240, a communication manager 242, configuration/radio settings 250, a device status log 252, and data 254. The device status log 252 includes a list of events specific to the access point 200. The events may include a log of normal events and error events, such as memory state, restart or restart events, crash events, cloud disconnect with self-recovery events, low link speed or link speed swing events, ethernet port state, ethernet interface packet errors, upgrade failure events, firmware upgrade events, configuration changes, etc., and time and date stamps for each event. The logging controller 255 determines the logging level of the device based on instructions from the NMS 130. Data 254 may store any data used and/or generated by access point 200, including data collected from UE 148, such as data used to calculate one or more SLE metrics, which is transmitted by access point 200 for cloud-based management of wireless network 106A by NMS 130.

Input/output (I/O) 210 represents physical hardware components, such as buttons, displays, etc., that enable interaction with a user. Although not shown, memory 212 typically stores executable software for controlling a user interface with respect to inputs received via I/O210. Communication manager 242 includes program code that, when executed by processor(s) 206, allows access point 200 to communicate with UE 148 and/or network(s) 134 via any interface(s) 230 and/or 220A-220C. Configuration settings 250 include any device settings for access point 200, such as radio settings for each of wireless interface(s) 220A-220C. These settings may be manually configured or may be monitored and managed remotely by NMS130 to optimize wireless network performance on a periodic basis (e.g., hourly or daily).

As described herein, AP device 200 may measure network data from status log 252 and report it to NMS130. Network data can include event data, telemetry data, and/or other SLE related data. The network data may include various parameters that indicate the performance and/or status of the wireless network. The parameters may be measured and/or determined by one or more of the UE devices in the wireless network and/or by one or more APs. NMS130 may determine one or more SLE metrics based on SLE related data received from APs in the wireless network and store the SLE metrics as network data 137 (fig. 1A). NMS130 may also update time graph database 138 (fig. 1A) of the network to include telemetry data received from APs in the wireless network over time, or at least entity and connectivity information extracted from the telemetry data.

Fig. 3 is a block diagram of an example Network Management System (NMS) 300 configured to provide troubleshooting for a site in accordance with one or more techniques of the present disclosure. NMS 300 may be used to implement NMS130 in fig. 1A-1B, for example. In such an example, NMS 300 is responsible for monitoring and managing one or more wireless networks 106A-106N at sites 102A-102N, respectively.

NMS 300 includes a communication interface 330, one or more processors 306, a user interface 310, memory 312, and database 318. The various elements are coupled together via a bus 314 on which they may exchange data and information. In some examples, NMS 300 receives data from one or more of client device 148, AP 142, switch 146, and other network nodes (e.g., router 187 of fig. 1B) within network 134, which can be used to calculate one or more SLE metrics and/or update time map database 317. The NMS 300 analyzes the data for cloud-based management of the wireless networks 106A through 106N. The received data is stored as network data 316 in a database 318 and telemetry data included in the received data or at least entity and connection information extracted from the telemetry data is stored in a time diagram database 317 in the database 318. In some examples, NMS 300 may be part of another server shown in fig. 1A or part of any other server.

The processor(s) 306 execute software instructions, such as those used to define software or a computer program, stored to a computer-readable storage medium (e.g., memory 312), such as a non-transitory computer-readable medium including a storage device (e.g., a disk drive or optical disk drive) or memory (e.g., flash memory or random access memory) or any other type of volatile or non-volatile memory, that stores instructions to cause the one or more processors 306 to perform the techniques described herein.

The communication interface 330 may comprise, for example, an ethernet interface. Communication interface 330 couples NMS300 to a network and/or the internet, such as any network(s) 134 and/or any local area network shown in fig. 1A. Communication interface 330 includes a receiver 332 and a transmitter 334 by which nms300 receives/transmits data and information to/from client device 148, AP 142, switch 146, servers 110, 116, 122, 128, and/or any other network node, device, or system forming part of network system 100, such as shown in fig. 1A. In some scenarios described herein, where network system 100 includes a "third party" network device owned by and/or associated with an entity other than NMS300, NMS300 does not receive, collect, or otherwise access network data from the third party network device.

The data and information received by NMS 300 may include, for example, telemetry data, SLE-related data, or event data received from one or more client devices AP 148, AP 142, switch 146, or other network nodes (e.g., router 187 of fig. 1B) that NMS 300 uses to remotely monitor the performance of wireless networks 106A-106N and application sessions from client devices to cloud-based application servers. NMS 300 may also send data to any network device, such as client device 148, AP 142, switch 146, other network nodes within network 134, administrator device 111, via communication interface 330 to remotely manage wireless networks 106A through 106N and portions of the wired network.

Memory 312 includes one or more devices configured to store programming modules and/or data associated with the operation of NMS 300. For example, the memory 312 may include a computer-readable storage medium, such as a non-transitory computer-readable medium including a storage device (e.g., a disk drive or optical drive) or memory (such as flash memory or random access memory) or any other type of volatile or non-volatile memory, that stores instructions to cause the one or more processors 306 to perform the techniques described herein.

In this example, memory 312 includes an API 320, SLE module 322, virtual Network Assistant (VNA)/AI engine 350, and Radio Resource Manager (RRM) 360. In accordance with the disclosed technology, the VNA/AI engine 350 includes a site troubleshooting engine 352, the site troubleshooting engine 352 constructing an application session-specific topology of the particular application session based on the data of the particular application session retrieved from the time graph database 317. In some examples, the site troubleshooting engine 352 applies the ML model 380 to the network data 316 and/or the time graph database 317 to perform troubleshooting of the particular application session by identifying root causes of network problems at one or more of the subset of network devices involved in the particular application session. NMS 300 may also include any other programming modules, software engines, and/or interfaces configured for remote monitoring and management of wireless networks 106A-106N and portions of wired networks, including remote monitoring and management of any AP 142/200, switch 146, or other network device, such as router 187 of fig. 1B.

SLE module 322 allows for the setting and tracking of SLE metrics thresholds for each network 106A-106N. SLE module 322 also analyzes SLE related data collected by the APs (e.g., any AP 142 from the UE in each wireless network 106A-106N). For example, APs 142A-1 through 142A-N collect SLE-related data from UEs 148A-1 through 148A-N currently connected to wireless network 106A. This data is sent to NMS 300, which NMS 300 is executed by SLE module 322 to determine one or more SLE metrics for each UE 148A-1 through 148A-N currently connected to wireless network 106A. In addition to any network data collected by one or more APs 142A-1 through 142A-N in wireless network 106A, this data is also sent to NMS 300 and stored in database 318 as, for example, network data 316.

RRM engine 360 monitors one or more metrics for each site 102A-102N to learn and optimize the RF environment for each site. For example, RRM engine 360 can monitor coverage and capacity SLE metrics for wireless network 106 at sites 102 to identify potential problems with SLE coverage and/or capacity in wireless network 106 and adjust radio settings for access points at each site to address the identified problems. For example, RRM engine 360 may determine the channel and transmit power distribution across all APs 142 in each network 106A-106N. For example, RRM engine 360 may monitor events, power, channels, bandwidth, and the number of clients connected to each AP. RRM engine 360 can also automatically change or update the configuration of one or more APs 142 at station 102 in order to improve coverage and capacity SLE metrics, thereby providing an improved wireless experience for the user.

The VNA/AI engine 350 analyzes data received from the network devices as well as its own data to identify when an unexpected or abnormal state is encountered at one of the network devices. For example, the VNA/AI engine 350 can identify the root cause of any unexpected or abnormal state, such as any worse SLE metric(s) indicative of connectivity problems at one or more network devices. Further, the VNA/AI engine 350 can automatically invoke one or more corrective actions directed to resolving the identified root cause(s) of the one or more bad SLE metrics. Examples of corrective actions that may be automatically invoked by VNA/AI engine 350 may include, but are not limited to, invoking RRM 360 to restart one or more APs, adjusting/modifying transmit power of a particular radio in a particular AP, adding an SSID configuration to a particular AP, changing channels on an AP or set of APs, and the like. Corrective actions may also include restarting the switch and/or router, invoking a download of new software to the AP, switch or router, etc. These corrective actions are given for illustrative purposes only and the disclosure is not limited in this respect. If automatic corrective actions are not available or do not adequately address the root cause, the VNA/AI engine 350 can proactively provide notifications that include recommended corrective actions to be taken by IT personnel (e.g., a site using the administrator device 111 or a network administrator) to address network errors.

In accordance with one or more techniques of this disclosure, the VNA/AI engine 350 may be configured to identify troubleshooting problems for multiple deployments at the site. For a particular site, the site troubleshooting engine 352 may identify a troubleshooting problem for each of a plurality of deployments (e.g., two or more WAN deployments, wireless deployments, or wired deployments). The VNA/AI engine 350 may determine the troubleshooting problem based on network data received from network devices (e.g., the client device 148, the AP device 142, the switch 146, and other network nodes such as the router 187 of fig. 1B). The site troubleshooting engine 352 may enable visualization of the troubleshooting problem, including color coding, icons, or other indicia of the troubleshooting problem at the site.

The time map database 317 is configured to store network connectivity and entity information extracted from historical telemetry data collected by the NMS 300 from client devices 148, APs 142, switches 146, and/or other network nodes within the network 134. Connectivity information may represent different types of connections, including wireless, wired, and logical links, such as peer-to-peer paths for SD-WAN devices or IPSec tunnels, such as router 187 from SD-WAN 177 of fig. 1B. The entity information may represent different kinds of network devices including client devices, AP devices, switches, other network nodes such as routes and gateways, third party network devices, and applications running on the network devices. NMS 300 uses connectivity and entity information to update time graph database 317, where the graph represents the network topology.

The site troubleshooting engine 352 may analyze the network data 316 of the subset of network devices related to the troubleshooting problem to identify the root cause of the site troubleshooting problem. More specifically, the site troubleshooting engine 352 may analyze event data included in the network device 316 or derived from the network device 316 to determine if further network problems exist. In some scenarios, the site troubleshooting engine 352 may apply at least a portion of the network data 316 to the ML model 380 to determine the root cause of the troubleshooting problem.

The site troubleshooting engine 352 may use different data for each of the different deployments deployed at the site to identify the network problem and/or the root cause of the network problem. For example, the site troubleshooting engine 352 may analyze the event data and/or telemetry data to identify a troubleshooting problem based on one or more of the data for wired deployment, the data for wireless deployment, or the data for WAN deployment. The site troubleshooting engine 352 may analyze AP health, radio health, pre-connection issues, RF issues, and/or configuration issues to identify network issues caused by the access points 142/200. The site troubleshooting engine 352 may analyze switch health, cable problems, lost Virtual Local Area Networks (VLANs), congestion, and/or configuration problems to identify network problems caused by the switch 146. Site troubleshooting engine 352 may analyze gateway health, WAN links, and/or configuration issues to identify network issues caused by routers or gateways 187A, 187B of SD-WAN 177.

The site troubleshooting engine 352 may generate data representing a user interface to provide to a user using the administrator device 111, such as a site or network administrator, including color coding, icons, or other indicia of troubleshooting problems at the site. In response to user input selecting an icon indicating a troubleshooting problem, the site troubleshooting engine 352 may further generate a troubleshooting user interface for the troubleshooting problem, or may redirect the user to customer insight or recommended actions.

In some examples, ML model 380 may include a supervised ML model trained using training data including pre-collected, labeled network data received from network devices (e.g., client devices, APs, switches, and/or other network nodes) to identify root causes of troubleshooting problems at one or more deployments at a site. The supervised ML model may include one of logistic regression, naive bayes, support Vector Machines (SVMs), etc. In other examples, ML model 380 may include an unsupervised ML model. Although not shown in fig. 3, in some examples, database 318 may store training data and VNA/AI engine 350 or a dedicated training module may be configured to train ML model 380 based on the training data to determine appropriate weights across one or more features of the training data.

In the event that a troubleshooting problem is detected at the site, the site troubleshooting engine 352 generates data representing a user interface for providing to a user using the administrator device 111, such as a site or network administrator, visualization of the troubleshooting problem including color coding, icons, or other indicia of the troubleshooting problem. In some examples, the VNA/AI engine 350 may determine the recommended action based on the detected troubleshooting problem and/or the root cause determined for the detected troubleshooting problem. The VNA/AI engine 350 may output a notification of the troubleshooting problem and/or the root cause of the troubleshooting problem via one or more of the user interface 310, the API 320, the network hook, or the email via the communication interface 330 for display on the administrator device 111 of the administrator.

In some examples, a site or network administrator using administrator device 111, for example, may initiate troubleshooting the site via conversation assistant engine 356. Dialog assistant engine 356 may be configured to process user input such as text strings and generate a response. In some examples, dialog assistant engine 356 may include one or more natural language processors configured to process user input. Conversation assistant engine 356 can be configured to conduct chat conversations that simulate the manner in which humans act as conversation partners, which can help simplify and/or improve administrator satisfaction with monitoring and controlling the network.

In accordance with one or more techniques of this disclosure, conversation assistant engine 356 can generate a conversation assistant configured to receive user input. In particular use cases, an administrator may input queries for particular network devices and/or particular application sessions into dialog assistant engine 356 via administrator device 111.

For example, the conversation assistant can receive a string that indicates a site (e.g., "troubleshoot a site cost Office," where "cost Office" indicates the site). Dialogue assistant engine 356 may determine the site and the particular network device deployed based on user input and determine one or more troubleshooting problems at the site. After identifying a particular site, the site troubleshooting engine 352 may identify troubleshooting problems at various deployments at the site based on the data of the particular application session retrieved from the time graph database 317. The site troubleshooting engine 352 may generate data representing the troubleshooting problem for presentation within the conversation assistant. The visualization may include color coding, icons, or other indicia of troubleshooting problems at the site deployment determined by the site troubleshooting engine 352 based on the temporal data stored as the network data 316 and/or the temporal map database 317. In this example, an administrator using administrator device 111 may interact with the troubleshooting problems presented within the conversation assistant to select the troubleshooting problems. In response to selection of the troubleshooting problem, the dialog assistant engine 356 may also generate a troubleshooting user interface for the troubleshooting problem to present within the dialog assistant. Additionally or alternatively, the conversation assistant engine 356 can redirect the user to a network device specific client insight or recommended action user interface.

The techniques of the present invention provide one or more technical advantages and practical applications. For example, these techniques enable site troubleshooting by identifying connection problems for each deployment. For example, the site troubleshooting engine 352 may access not only wireless data to determine problems at the site, but also wired data and/or WAN data. In this way, the site troubleshooting engine 352 may determine wired issues and/or WAN issues, which may help reduce the amount of time an administrator spends on site troubleshooting. Further, the site troubleshooting engine 352 can troubleshoot WAN, wireless, and wired deployments simultaneously, and can identify root causes for all deployment types faster than systems that use only data from wireless deployments to identify root causes. In addition, the site troubleshooting engine 135 can recommend actions in the conversation assistance to quickly solve the problem(s).

Fig. 4 illustrates an example User Equipment (UE) device 400. The example UE device 400 shown in fig. 4 may be used to implement any UE 148 as shown and described herein with reference to fig. 1A. UE device 400 may include any type of wireless client device and the disclosure is not limited in this respect. For example, UE device 400 may include a mobile device such as a smart phone, tablet or laptop computer, personal Digital Assistant (PDA), wireless terminal, smart watch, smart ring, or any other type of mobile or wearable device. In accordance with the techniques described in this disclosure, the UE 400 may also include a wired client device, such as an IoT device, e.g., a printer, security sensor or device, environmental sensor, or any other device connected to a wired network and configured to communicate over one or more wireless networks.

UE device 400 includes wired interface 430, wireless interfaces 420A-420C, one or more processors 406, memory 412, and user interface 410. The various elements are coupled together via a bus 414 on which the various elements may exchange data and information. The wired interface 430 represents a physical network interface and includes a receiver 432 and a transmitter 434. If desired, the wired interface 430 may be used to couple the UE 400 directly or indirectly to a wired network device within a wired network, such as one of the switches 146 of FIG. 1A, via a cable, such as one of the Ethernet cables 144 of FIG. 1A.

The first, second, and third wireless interfaces 420A, 420B, and 420C include receivers 422A, 422B, and 422C, respectively, each including a receive antenna via which the UE 400 may receive wireless signals from a wireless communication device, such as the AP 142 of fig. 1A, the AP 200 of fig. 2, other UEs 148, or other devices configured for wireless communication. The first, second, and third wireless interfaces 420A, 420B, and 420C also include transmitters 424A, 424B, and 424C, respectively, each including a transmit antenna through which the UE 400 may transmit wireless signals to wireless communication devices, such as the AP 142 of fig. 1A, the AP 200 of fig. 2, other UEs 148, and/or other devices configured for wireless communication. In some examples, the first wireless interface 420A may include a Wi-fi802.11 interface (e.g., 2.4GHz and/or 5 GHz), and the second wireless interface 420B may include a bluetooth interface and/or a bluetooth low energy interface. The third wireless interface 420C may comprise, for example, a cellular interface through which the UE device 400 may connect to a cellular network.

The processor(s) 406 execute software instructions, such as those used to define software or a computer program, stored to a computer-readable storage medium (e.g., memory 412), such as a non-transitory computer-readable medium including a storage device (e.g., a disk drive or optical disk drive) or memory (e.g., flash memory or random access memory) or any other type of volatile or non-volatile memory, that stores instructions to cause the one or more processors 406 to perform the techniques described herein.

Memory 412 includes one or more devices configured to store programming modules and/or data associated with the operation of UE 400. For example, memory 412 may include a computer-readable storage medium, such as a non-transitory computer-readable medium including a storage device (e.g., a disk drive or optical drive) or memory (such as flash memory or random access memory) or any other type of volatile or non-volatile memory, that stores instructions to cause one or more processors 406 to perform the techniques described herein.

In this example, memory 412 includes an operating system 440, applications 442, communication modules 444, configuration settings 450, and data 454. The communication module 444 includes program code that, when executed by the processor(s) 406, enables the UE 400 to communicate using any of the wired interface(s) 430, wireless interfaces 420A-420B, and/or cellular interface 450C. Configuration settings 450 include any device settings for UE 400 settings for each of wireless interface(s) 420A-420B and/or cellular interface 420C.

The data 454 may include, for example, a status/error log that includes a list of events specific to the UE 400. The event may include a log of both normal and error events according to a logging level based on instructions from NMS130. Data 454 may include any data used and/or generated by UE 400, such as data used to calculate one or more SLE metrics or identify relevant behavior data, which is collected by UE 400 and sent directly to NMS130 or to any AP 142 in wireless network 106 for further transmission to NMS130.

As described herein, UE 400 may measure network data from data 454 and report it to NMS130. Network data can include event data, telemetry data, and/or other SLE related data. The network data may include various parameters that indicate the performance and/or status of the wireless network. NMS130 may determine one or more SLE metrics based on SLE related data received from UEs or client devices in the wireless network and store the SLE metrics as network data 137 (fig. 1A). NMS130 may also update time graph database 138 (fig. 1A) of the network to include telemetry data received from UEs or client devices in the wireless network over time, or at least entity and connection information extracted from the telemetry data.

NMS agent 456 is a software agent of NMS130 installed on UE 400. In some examples, NMS agent 456 may be implemented as a software application running on UE 400. NMS agent 456 gathers information from UE 400 that includes detailed client device attributes, including insight into the roaming behavior of UE 400. This information provides insight into the client roaming algorithm, as roaming is a decision by the client device. In some examples, NMS agent 456 may display client device attributes on UE 400. NMS agent 456 transmits client device attributes to NMS130 via the AP device to which UE 400 is connected. NMS agent 456 may be integrated into the custom application or as part of the positioning application. NMS agent 456 may be configured to identify the device connection type (e.g., cellular or Wi-Fi) and the corresponding signal strength. For example, NMS agent 456 identifies an access point connection and its corresponding signal strength. NMS agent 456 may store information specifying APs identified by UE 400 and their corresponding signal strengths. The NMS agent 456 or other element of the UE 400 also gathers information about which APs the UE 400 is connected to, which information also indicates which APs the UE 400 is not connected to. The NMS agent 456 of the UE 400 sends this information to the NMS130 through the AP to which it is connected. In this way, the UE 400 transmits not only information about the APs to which the UE 400 is connected, but also information about other APs identified and not connected by the UE 400 and their signal strengths. The AP in turn forwards this information to the NMS, including information about other APs that UE 400 identifies in addition to itself. This additional level of granularity enables NMS130, and ultimately the network administrator, to better determine the Wi-Fi experience directly from the perspective of the client device.

In some examples, NMS agent 456 further enriches client device data utilized in the service class. For example, NMS agent 456 may override the basic fingerprint identification to provide additional details to attributes such as device type, manufacturer, and different versions of the operating system. In the detailed client properties, NMS130 may display radio hardware and firmware information of UE 400 received from NMS client agent 456. The more details the NMS agent 456 can draw, the better the VNA/AI engine gets in terms of advanced device classification. The VNA/AI engine of NMS130 is constantly learning and becomes more accurate in its ability to distinguish device-specific problems or wide range of device problems, for example, specifically identifying that a particular OS version is affecting certain clients.

In some examples, NMS agent 456 may cause user interface 410 to display a prompt prompting an end user of UE 400 to enable location permissions before NMS agent 456 can report the location of the device, client information, and network connection data to the NMS. NMS agent 456 will then begin reporting connection data as well as location data to the NMS. In this way, the end user of the client device may control whether NMS agent 456 is enabled to report client device information to the NMS.

Fig. 5 is a block diagram illustrating an example network node 500 configured in accordance with the techniques described herein. In one or more examples, network node 500 implements a device or server attached to network 134 of fig. 1A, such as switch 146, AAA server 110, DHCP server 116, DNS server 122, web server 128, etc., or another network device supporting one or more of wireless network 106, wired LAN175, or SD-WAN 177 of fig. 1B, or one or more of data center 179 of fig. 1B, such as router 187.

In this example, network node 500 includes a wired interface 502 (e.g., an ethernet interface), one or more processors 506, input/output 508 (e.g., display, buttons, keyboard, keypad, touch screen, mouse, etc.), and memory 512 coupled together via a bus 514 over which various elements may exchange data and information. A wired interface 502 couples the network node 500 to a network, such as an enterprise network. Although only one interface is shown as an example, a network node may and typically does have multiple communication interfaces and/or multiple communication interface ports. The wired interface 502 includes a receiver 520 and a transmitter 522.

Memory 512 stores executable software applications 532, operating system 540, and data 530. The data 530 may include a system log and/or an error log storing event data (including behavior data) of the network node 500. In examples where network node 500 includes a "third party" network device, the same entity does not own the AP or the wired client-side device and network node 500, or cannot own or access both. Thus, in examples where network node 500 is a third party network device, NMS130 does not receive, collect, or otherwise access network data from network node 500.

In examples where network node 500 includes a server, network node 500 may receive data and information via receiver 520, including, for example, operation-related information, such as registration requests, AAA services, DHCP requests, simple Notification Service (SNS) lookup, and web page requests, and send data and information via transmitter 522, including, for example, configuration information, authentication information, web page data, and the like.

In examples where network node 500 includes a wired network device, network node 500 may connect to one or more APs or other wired client devices within the wired network edge, such as IoT devices, via wired interface 502. For example, network node 500 may include a plurality of wired interfaces 502 and/or wired interfaces 502 may include a plurality of physical ports to connect to a plurality of APs or other wired client devices within a site via respective ethernet cables. In some examples, each AP or other wired client-side device connected to network node 500 may access a wired network via wired interface 502 of network node 500. In some examples, one or more APs or other wired client-side devices connected to network node 500 may each obtain power from network node 500 via a respective power over ethernet cable and power over ethernet (PoE) port of wired interface 502.

In examples where network node 500 includes a session-based router employing a stateful, session-based routing scheme, network node 500 may be configured to independently perform path selection and traffic engineering. Using session-based routing may enable network node 500 to avoid using a centralized controller, such as an SDN controller, to perform path selection and traffic engineering, and to avoid using tunnels. In some examples, network node 500 may implement session-based routing as Secure Vector Routing (SVR) provided by a prospective blogging network company. Where network node 500 includes a session-based router (e.g., router 187A of fig. 1B) that acts as a network gateway for a site of an enterprise network, network node 500 may be at an underlying physical WAN (e.g., SD-WAN 177 of fig. 1B) and one or more other session-based routers (e.g., router 187B of fig. 1B) that act as network gateways for other sites of the enterprise network. The network node 500 operating as a session-based router may collect data at the peer path level and report the peer path data to the NMS130.

In examples where network node 500 includes a packet-based router, network node 500 may employ a packet-based or flow-based routing scheme to forward packets according to defined network paths established, for example, by a centralized controller that performs path selection and traffic engineering. Where network node 500 includes a packet-based router (e.g., router 187A of fig. 1B) that serves as a network gateway for a site of an enterprise network, network node 500 may establish a plurality of tunnels (e.g., logical path 189 of fig. 1B) with one or more other packet-based routers (e.g., router 187B of fig. 1B) that serve as network gateways for other sites of the enterprise network through an underlying physical WAN (e.g., SD-WAN 177 of fig. 1B). The network node 500 operating as a packet-based router may collect data at the tunnel level and the tunnel data may be retrieved by the NMS130 via an API or open configuration protocol, or the tunnel data may be reported to the NMS130 by the NMS agent 544 or other module running on the network node 500.

The data collected and reported by the network node 500 may include periodically reported data and event driven data. The network node 500 is configured to collect logical path statistics via Bidirectional Forwarding Detection (BFD) detection and data extracted from messages and/or counters of logical path (e.g., peer-to-peer path or tunnel) classes. In some examples, the network node 500 is configured to collect statistics and/or sample other data according to a first periodic interval (e.g., every 3 seconds, every 5 seconds, etc.). The network node 500 may store the collected and sampled data as path data, e.g., in a buffer. In some examples, NMS agent 544 may periodically create packets of statistics according to a second periodic interval (e.g., every 3 minutes). The collected and sampled data that is periodically reported in the statistics packet may be referred to herein as "oc-stats".

In some examples, the statistics packet may also include details regarding clients and associated client sessions connected to the network node 500. NMS agent 544 may then report the statistics packet to NMS130 in the cloud. In other examples, NMS130 may request, retrieve, or otherwise receive statistics packets from network node 500 via an API, an open configuration protocol, or another communication protocol. The statistics packet created by NMS agent 544 or another module of network node 500 may include statistics and data samples identifying the header of network node 500 and each logical path from network node 500. In other examples, NMS agent 544 reports event data to NMS130 in the cloud in response to certain events occurring at network node 500 when the event occurs. Event driven data may be referred to herein as "oc-events".

Fig. 6 illustrates an example user interface 600 of a network management system for troubleshooting a site in accordance with one or more techniques of the present disclosure. FIG. 6 illustrates an example conversation assistant user interface 600 that includes a query or user input 610 from an administrator via the administrator device 111 indicating a site (e.g., "Mist Office") and a response or output 612, 614 generated by the conversation assistant engines 136, 356. In the example of fig. 6, the user input 610 of the conversation assistant user interface 600 includes a string indicating an application and a device identifier (i.e., "troubleshooting a site cost Office," where "cost Office" indicates the site). The response 612 within the dialog assistant user interface 600 includes a string stating "CHECKING SITE MIST office, here is what I found on Sep, ^th" (check site best office, which is what i found on day 9, 30). Further, the dialog assistant user interface 600 presents the output 614 as a visualization of at least the first troubleshooting problem, the second troubleshooting problem, and the third troubleshooting problem. In the example of fig. 6, output 614 includes a "WAN" string indicating "no significant problem found" for a first troubleshooting problem for a WAN deployment, a "wireless" string indicating "client(s) on site experiencing ethernet error problem(s)" for a second troubleshooting problem for a wireless deployment, and a "wired" string indicating "switch(s) on site experiencing switch disconnection problem(s)" for a third troubleshooting problem for a wired deployment.

In the example of fig. 6, the dialog assistant user interface 600 presents the output 614 as a visualization of the first troubleshooting problem, the second troubleshooting problem, and the third troubleshooting problem in the ordered list. For example, conversation assistant user interface 600 presents output 614 as an ordered list of second troubleshooting problems for wireless deployments (e.g., "client(s) on site experienced ethernet error problem (s)), then third troubleshooting problems for wired deployments (e.g.," switch(s) on site experienced switch disconnect problem (s)), and then first troubleshooting problems for WAN deployments (e.g., "no significant problem found").

FIG. 7 is a flowchart illustrating example operations of site troubleshooting in accordance with one or more techniques of the present disclosure. The example operation of fig. 7 is described herein with reference to NMS 300 of fig. 3. In other examples, the operations of fig. 7 may be performed by other computing devices, such as NMS130 of fig. 1A-1B.

NMS 300 may receive a query identifying a site of a plurality of sites (702). For example, a human administrator may input a query using administrator device 111. In some examples, NMS 300 may apply one or more natural language processors configured to process user input. The site may include one or more of a WAN deployment, a wireless deployment, or a wired deployment. The WAN deployment may include an intermediary network communicatively coupling the wireless deployment and the wired deployment to the application service. The wireless deployment may include a wireless access point configured to support one or more of Wi-Fi or bluetooth. A wired deployment may include one or more network devices in a collection of network devices at a site that are connected using physical cables.

The NMS 300 may determine a first set of troubleshooting problems for WAN deployment of the set of network devices at the site, a second set of troubleshooting problems for wireless deployment of the set of network devices at the site, and a third set of troubleshooting problems for wired deployment of the set of network devices at the site based on the network data received from the plurality of network devices (704). For example, NMS 300 may determine one or more of one or more client problems, connectivity problems, or device health problems for WAN deployments. Similarly, NMS 300 may determine one or more of one or more client problems, connectivity problems, or device health problems for wireless deployments and/or wired deployments.

NMS 300 may determine a first troubleshooting problem from a first set of troubleshooting problems for the WAN deployment, a second troubleshooting problem from a second set of troubleshooting problems for the wireless deployment, and a third troubleshooting problem from a third set of troubleshooting problems for the wired deployment based on the user experience metrics (706). For example, NMS 300 may determine that the first troubleshooting problem has the highest impact (e.g., highest SLE score value) on the user experience of the first set of troubleshooting problems for the WAN deployment based on the user experience metrics (e.g., SLE scores). Similarly, NMS 300 may determine that the second troubleshooting problem has the highest impact on the user experience of the second set of troubleshooting problems for the wireless deployment based on the user experience metrics and/or that the third troubleshooting problem has the highest impact on the user experience of the third set of troubleshooting problems for the wired deployment based on the user experience metrics. In some examples, the user experience metrics include a service level desire (SLE) score.

In some examples, NMS 300 may determine an ordered list of a first troubleshooting problem for WAN deployments, a second troubleshooting problem for wireless deployments, and a third troubleshooting problem for wired deployments. For example, NMS 300 may determine an ordered list of a first troubleshooting problem for WAN deployments, a second troubleshooting problem for wireless deployments, and a third troubleshooting problem for wired deployments based on user experience metrics.

NMS 300 may generate data representing a user interface for presentation on an administrator device, the user interface including a visualization of at least a first troubleshooting problem, a second troubleshooting problem, and a third troubleshooting problem (708). For example, NMS 300 may generate data representing the user interface shown in fig. 6. In some examples, NMS 300 may generate a user interface to include in the ordered list a visualization of the first troubleshooting problem, the second troubleshooting problem, and the third troubleshooting problem.

In response to receiving user input selecting an icon representing a troubleshooting problem, NMS 300 may perform a root cause analysis to determine a root cause for one or more of the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem. In this example, NMS 300 may generate data representing a dialog assistant user interface comprising a platform configured to receive a query identifying a site, present a visual user interface comprising one or more of a first troubleshooting problem, a second troubleshooting problem, or a third troubleshooting problem, and receive user input interacting with the user interface. The NMS 300 may also generate data representing a troubleshooting user interface for presentation on an administrator device, the troubleshooting user interface including at least one indication of a root cause of at least one connectivity problem at a network device. In some examples, NMS 300 may identify actions for one or more of the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem. The NMS 300 may identify a number of a plurality of pending actions for one or more of the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem.

The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. The various features described as modules, units, or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices or other hardware devices. In some cases, various features of the electronic circuit may be implemented as one or more integrated circuit devices, such as an integrated circuit chip or chipset.

If implemented in hardware, the present disclosure may be directed to a device such as a processor or an integrated circuit device such as an integrated circuit chip or chipset. Alternatively or additionally, if implemented in software or firmware, the techniques may be realized at least in part by a computer-readable data storage medium comprising instructions that, when executed, cause a processor to perform one or more of the methods described above. For example, a computer-readable data storage medium may store such instructions for execution by a processor.

The computer readable medium may form part of a computer program product, which may include packaging material. The computer-readable medium may include computer data storage media such as Random Access Memory (RAM), read Only Memory (ROM), non-volatile random access memory (NVRAM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory, magnetic or optical data storage media, and the like. In some examples, an article of manufacture may comprise one or more computer-readable storage media.

In some examples, the computer-readable storage medium may include a non-transitory medium. The term "non-transitory" may mean that the storage medium is not embodied in a carrier wave or propagated signal. In some examples, a non-volatile storage medium may store data that may change over time (e.g., in RAM or cache).

The code or instructions may be software and/or firmware executed by processing circuitry including one or more processors, such as one or more Digital Signal Processors (DSPs), general purpose microprocessors, application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Furthermore, in some aspects, the functionality described in this disclosure may be provided within software modules or hardware modules.

Claims

1. A system for identifying network problems, the system comprising:

A memory storing network data received from a plurality of network devices arranged at a plurality of sites; and

One or more processors coupled to the memory and configured to:

receiving a query identifying a site of the plurality of sites;

Determining, based on network data received from the plurality of network devices, a first set of troubleshooting problems for Wide Area Network (WAN) deployment of a set of network devices at the site, a second set of troubleshooting problems for wireless deployment of the set of network devices at the site, and a third set of troubleshooting problems for wired deployment of the set of network devices at the site;

Determining, based on user experience metrics, a first troubleshooting problem from the first set of troubleshooting problems for the WAN deployment, a second troubleshooting problem from the second set of troubleshooting problems for the wireless deployment, and a third troubleshooting problem from the third set of troubleshooting problems for the wired deployment; and

Data representing a user interface for presentation on an administrator device is generated, the user interface including a visualization of at least the first troubleshooting problem, the second troubleshooting problem, and the third troubleshooting problem.

2. The system according to claim 1,

Wherein, to determine the first set of troubleshooting problems for the WAN deployment, the one or more processors are configured to determine one or more of a client problem, connectivity problem, or device health problem for the WAN deployment;

wherein, to determine the second set of troubleshooting problems for the wireless deployment, the one or more processors are configured to determine one or more of a client problem, connectivity problem, or device health problem for the wireless deployment; and

Wherein to determine the third set of troubleshooting problems for the wired deployment, the one or more processors are configured to determine one or more of a client problem, connectivity problem, or device health problem for the wired deployment.

3. The system according to claim 1,

Wherein, to determine the first troubleshooting problem from the first set of troubleshooting problems for the WAN deployment, the one or more processors are configured to determine, based on the user experience metrics, that the first troubleshooting problem has a highest impact on a user experience of the first set of troubleshooting problems for the WAN deployment;

wherein, to determine the second troubleshooting problem from the second set of troubleshooting problems for the wireless deployment, the one or more processors are configured to determine, based on the user experience metrics, that the second troubleshooting problem has the highest impact on the user experience of the second set of troubleshooting problems for the wireless deployment; and

Wherein, to determine the third troubleshooting problem from the third set of troubleshooting problems for the wired deployment, the one or more processors are configured to determine, based on the user experience metrics, that the third troubleshooting problem has the highest impact on the user experience of the third set of troubleshooting problems for the wired deployment.

4. The system of claims 1-3, wherein the one or more processors are configured to:

Determining an ordered list of the first troubleshooting problem for the WAN deployment, the second troubleshooting problem for the wireless deployment, and the third troubleshooting problem for the wired deployment based on the user experience metrics,

Wherein, to generate the data representing the user interface for presentation on the administrator device, the one or more processors are configured to generate the data representing the user interface to include the visualizations of the first, second, and third troubleshooting problems in the ordered list.

5. The system of any of claims 1-3, wherein the user experience metric comprises a service level desire score.

6. A system according to any one of claims 1 to 3, wherein the one or more processors are configured to perform root cause analysis to determine a root cause of one or more of the first, second or third troubleshooting problems.

7. The system of any of claims 1 to 3, wherein the one or more processors are configured to: generating data representing a dialog assistant user interface comprising a platform configured to receive a query identifying the site; presenting the user interface comprising a visualization of at least the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem; and receiving user input interacting with the user interface.

8. A system according to any one of claims 1 to 3, wherein the one or more processors are configured to identify an action to remedy one or more of the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem.

9. A system according to any one of claims 1 to 3, wherein the one or more processors are configured to identify a number of a plurality of pending actions for one or more of the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem.

10. The system of any of claims 1-3, wherein the WAN deployment comprises an intermediary network communicatively coupling the wireless deployment and the wired deployment to an application service.

11. The system of any of claims 1-3, wherein the wireless deployment comprises one or more wireless access point devices configured to support one or more of Wi-Fi or bluetooth.

12. A system according to any one of claims 1 to 3, wherein the wired deployment comprises one or more network devices of the set of network devices at the site connected using a physical cable.

13. A method of identifying a network problem, the method comprising:

Receiving, by one or more processors, a query identifying a site of a plurality of sites;

Determining, by the one or more processors and based on network data received from a plurality of network devices disposed at the plurality of sites, a first set of troubleshooting problems for wide area network, WAN, deployment of a set of network devices at the sites, a second set of troubleshooting problems for wireless deployment of the set of network devices at the sites, and a third set of troubleshooting problems for wired deployment of the set of network devices at the sites;

Determining, by the one or more processors, a first troubleshooting problem from the first set of troubleshooting problems for the WAN deployment, a second troubleshooting problem from the second set of troubleshooting problems for the wireless deployment, and a third troubleshooting problem from the third set of troubleshooting problems for the wired deployment based on user experience metrics; and

Data representing a user interface for presentation on an administrator device is generated by the one or more processors, the user interface including a visualization of at least the first troubleshooting problem, the second troubleshooting problem, and the third troubleshooting problem.

14. The method of claim 13, wherein determining the first set of troubleshooting problems for the WAN deployment comprises: determining one or more of a client problem, connectivity problem, or device health problem for the WAN deployment;

wherein determining the second set of troubleshooting problems for the wireless deployment comprises: determining one or more of a client problem, connectivity problem, or device health problem for the wireless deployment; and

Wherein determining the third set of troubleshooting problems for the wired deployment comprises: one or more of a client problem, connectivity problem, or device health problem for the wired deployment is determined.

15. The method according to claim 13,

Wherein determining the first troubleshooting problem from the first set of troubleshooting problems for the WAN deployment comprises: determining, based on the user experience metrics, that the first troubleshooting problem has the highest impact on user experience of the first set of troubleshooting problems for the WAN deployment;

Wherein determining the second troubleshooting problem from the second set of troubleshooting problems for the wireless deployment comprises: determining, based on the user experience metrics, that the second troubleshooting problem has the highest impact on the user experience of the second set of troubleshooting problems for the wireless deployment; and

Wherein determining the third troubleshooting problem from the third set of troubleshooting problems for the wired deployment comprises: determining, based on the user experience metrics, that the third troubleshooting problem has the highest impact on the user experience of the third set of troubleshooting problems for the wired deployment.

16. The method of any of claims 13 to 15, further comprising:

Determining, by the one or more processors and based on the user experience metrics, an ordered list of the first troubleshooting problem for the WAN deployment, the second troubleshooting problem for the wireless deployment, and the third troubleshooting problem for the wired deployment,

Wherein generating the data representing the user interface for presentation on the administrator device comprises: generating the data representing the user interface to include the visualizations of the first, second, and third troubleshooting problems in the ordered list.

17. The method of any of claims 13 to 15, wherein the user experience metric comprises a service level desire score.

18. The method of any of claims 13 to 15, further comprising: root cause analysis is performed by the one or more processors to determine a root cause of one or more of the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem.

19. The method of any of claims 13 to 15, further comprising: generating, by the one or more processors, data representing a dialog assistant user interface comprising a platform configured to receive a query identifying the site; presenting the user interface comprising a visualization of at least the first troubleshooting problem, the second troubleshooting problem, or the third troubleshooting problem; and receiving user input interacting with the user interface.

20. A computer-readable storage medium encoded with instructions for causing one or more programmable processors to perform a process implemented by the system of any of claims 1-12.