US20180300190A1 - Mobile application service engine (mase) - Google Patents
Mobile application service engine (mase) Download PDFInfo
- Publication number
- US20180300190A1 US20180300190A1 US15/789,904 US201715789904A US2018300190A1 US 20180300190 A1 US20180300190 A1 US 20180300190A1 US 201715789904 A US201715789904 A US 201715789904A US 2018300190 A1 US2018300190 A1 US 2018300190A1
- Authority
- US
- United States
- Prior art keywords
- application
- airtime
- certain embodiments
- block
- mase
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
- G06F16/183—Provision of network file services by network file servers, e.g. by using NFS, CIFS
-
- G06F17/30203—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0231—Traffic management, e.g. flow control or congestion control based on communication conditions
- H04W28/0236—Traffic management, e.g. flow control or congestion control based on communication conditions radio quality, e.g. interference, losses or delay
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0289—Congestion control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/086—Load balancing or load distribution among access entities
- H04W28/0861—Load balancing or load distribution among access entities between base stations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/50—Service provisioning or reconfiguring
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W48/00—Access restriction; Network selection; Access point selection
- H04W48/20—Selecting an access point
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/04—Network management architectures or arrangements
- H04L41/046—Network management architectures or arrangements comprising network management agents or mobile agents therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/53—Network services using third party service providers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W88/00—Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
- H04W88/08—Access point devices
Definitions
- the present invention is directed to wireless communications, and more specifically to aspects of WiFi network architecture and services.
- FIG. 1 shows a high-level diagram that illustrates a cloud-controlled service-ready access point device architecture, according to certain embodiments.
- FIG. 2 illustrates the cloud-controlled service-ready access point device architecture of FIG. 1 in greater detail, according to certain embodiments.
- FIGS. 3A, 3B and 3C illustrate management of MASE applications, according to certain embodiments.
- FIG. 4 illustrates MASE libraries to support MASE applications installed in a given AP device, according to certain embodiments.
- FIG. 5 illustrates segregation of a portion of the AP file system to confine MASE applications, according to certain embodiments.
- FIG. 6 is a high-level network diagram showing aspects of distributing the consumption of airtime allocated to each connected wireless client device across a cycle time in a wireless communication network, according to certain embodiments.
- FIGS. 7A, 7B, 7C, and 7D are high-level flow charts showing aspects of an airtime calculation engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- FIG. 8 is a high-level network flow chart showing aspects of an airtime distribution engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- FIG. 9 is a high-level network flow chart showing aspects of a throttler for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- FIG. 10 is a high-level network flow chart showing aspects of a feedback engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- FIG. 11 is a high-level network diagram showing aspects of a distributed load balancing solution, according to certain embodiments.
- FIG. 12 is a high-level flow chart that illustrates some of the functions of a current AP that receives a probe request, according to certain embodiments.
- FIG. 13 is a high-level flow chart that illustrates some of the operations of the client load balance server, according to certain embodiments.
- FIG. 14 is a high-level flow chart that illustrates some of the features of the ranking operation for ranking the APs, according to certain embodiments.
- FIG. 15 is a graph that illustrates aspects of the ranking logic used for load balancing clients across APs in a wireless environment, according to certain embodiments.
- FIG. 16 is a high-level flow chart that illustrates some aspects of the decision logic engine used by the current AP, according to certain embodiments.
- FIG. 17 is a graph that illustrates aspects of the decision logic used for load balancing clients across APs in a wireless environment, according to certain embodiments.
- a mobile application services engine enables the deployment (in clusters or otherwise) of third party applications at access point devices (“AP”).
- AP access point devices
- such a deployment of third party applications can be managed remotely using a cloud implemented network management system (cloud-based NMS). Further, the NMS can remotely monitor the activities and status of the third party applications deployed in one or more APs associated with the cloud-based NMS.
- a wireless access point is a device that allows wireless devices (client devices) to connect to a wired network using Wi-Fi, or related standards.
- An enterprise grade AP device includes a chipset that is capable of connecting hundreds of wireless client devices to a wired network at the same time.
- third party applications are deployed on a given AP using software containers (e.g., Linux containers).
- software containers e.g., Linux containers.
- Such software containers provide a secure environment for the third party applications, while at the same time, protects the AP by confining a given third party application in an isolated file system as described in greater detail herein.
- the embodiments are not restricted to Linux containers.
- the type of containers may vary from implementation to implementation.
- such containers preserve the stability and quality of the AP without requiring the customer to override the AP's firmware and/or rebuild the AP's image files.
- a customer is any establishment or entity that owns or rents APs for deployment in the establishment/buildings.
- the customer can be either a retailer or a wireless service provider that deploys the access point device(s).
- applications can be grouped (clustered) such that applications of the same group share the same container in a given AP.
- application groups that are installed and run in separate containers can reduce interference and conflicts between the application groups.
- such third party applications in the containers have access to standard Linux services as well as access to advanced features provided by MASE through the MASE application interface (API) on a given AP.
- MASE applications have access to standard Linux services as well as access to advanced features provided by MASE through the MASE application interface (API) on a given AP.
- standard Linux system services are available to the MASE applications using Linux system calls.
- Advanced system services are available to the MASE applications through the MASE API.
- a MASE application includes configuration files, any libraries and other binaries needed to run the software application, bundled into the container.
- a given AP device shares the AP's operating system, CPU and memory with the MASE applications.
- a given AP device includes a MASE run-time management system to manage and monitor the MASE applications.
- the MASE run-time management system monitors the use of a given AP's resources by each MASE application to prevent hogging of resources by the MASE applications.
- the AP device in order for an AP device to host MASE applications and to host the MASE run-time management system, includes: 1) WLAN chipset that is capable of connecting at least about 50 wireless client devices at the same time to the wired network that is associated with the AP device (preferably, the WLAN chipset is capable of connecting at least 100 wireless client devices to the wired network that is associated with the AP device), 2) a stand-alone multi-core CPU (central processing unit) that is capable of hosting relevant WiFi related connection services for wireless client devices, as well as capable of hosting/supporting MASE applications, 3) high-speed and large RAM memory capable of hosting MASE applications on the AP device (e.g., RAM size ranges from approximately 1 gigabyte to approximately 4 gigabyte), 4) high-speed and large permanent storage (e.g., permanent storage size ranges from approximately 8 gigabyte to approximately 128 gigabyte) for supporting MASE applications and for storing large amounts of cached data to support better web-services delivery, and 5) dedicated network processor hardware that
- third party applications are installed on the one or more APs as MASE application packages using standard ipk packaging (Itsy Package) technology.
- Itsy Package standard ipk packaging
- the embodiments are not limited to ipk packaging technology.
- the type of packaging technology used may vary from implementation to implementation.
- An application package includes configuration files, software libraries and executable code, for example.
- FIG. 1 shows a high-level diagram that illustrates a cloud-controlled service-ready access point device architecture 100 , according to certain embodiments.
- FIG. 1 shows a plurality of cloud-controlled service-ready access point devices (AP) 102 a, . . . , AP 102 n and a cloud-based NMS 104 .
- AP cloud-controlled service-ready access point devices
- AP 102 a . . . AP 102 n are associated with the cloud-based NMS 104 in a cloud infrastructure that supports the plurality of cloud-controlled service-ready access point devices.
- Each of AP 102 a, . . . , AP 102 n includes a respective container 108 a, . . . , 108 n, which in turn includes a respective application 110 a, . . . , 110 n.
- the applications 110 a, . . . , 110 n are MASE applications because they are containerized at the respective APs.
- each AP device can include one container or multiple containers. Further, each container can contain one MASE application or multiple MASE applications. For example, applications can be grouped so that the applications of the same group can share the same container in a given AP. Application groups that are installed and run in separate containers can reduce interference and conflicts between the application groups.
- the cloud-based NMS 104 has the following functions with respect to the MASE applications: 1) deploy the MASE applications in the one or more APs, 2) manage installation of the one or more MASE applications, 3) upgrade the one or more MASE applications when needed, 4) monitor the running status of the one or more MASE applications, 5) configure the one or more MASE applications, and 6) maintain the one or more MASE applications (e.g., the access to syslog files of MASE applications are implemented through the cloud-based NMS).
- the APs use open source Linux (OpenWRT).
- third party developers use the MASE API and the MASE software development kit (SDK, which is based on the OpenWRT build environment) to develop the MASE application.
- the MASE SDK includes tools for third party developers to build MASE application packages and to download the application packages to the APs.
- the embodiments are not limited to the Linux OpenWRT operating system.
- the type of operating system used for the AP may vary from implementation to implementation.
- the MASE applications can be uploaded to the cloud-based NMS.
- the Cloud based NMS provides an interactive GUI (graphical user interface) to facilitate the upload process.
- cloud-based NMS 104 is the network management system that manages clusters of APs. Cloud-based NMS 104 is responsible for the full cycle deployment of third party applications such that the applications can be managed remotely via cloud-based NMS 104 without any need of performing administrative tasks locally on the APs.
- MASE applications can be effectively “walled-off” by confining them to a pre-allocated a sub-file system of limited size within the AP file system as explained in greater detail herein with reference to FIG. 5 , herein.
- FIG. 2 illustrates the cloud-controlled service-ready access point device architecture 200 in greater detail, according to certain embodiments.
- FIG. 2 shows only one of the APs of the plurality of APs.
- cloud-based NMS 204 includes a cloud-based controller 250 and MASE applications that can be downloaded and installed on AP device 202 .
- the MASE applications stored in NMS 204 include media applications 252 , analytics applications 254 , customized applications 256 , as well as various applications A-N in application marketplace 258 from which a customer can choose to download one or more applications A-N onto the AP device 202 .
- AP device (AP) 202 communicates with cloud-based NMS 204 through a secure tunnel in Internet 206 , for example.
- non-limiting examples of characteristics of AP device 202 that include operating system services 220 (e.g., Linux system services), web service 222 , deep packet inspection (DPI) service 224 , application configuration 226 , NMS events interface 228 , operating system calls 230 , MASE application programing interface 232 (MASE API), MASE application 210 (downloaded from cloud-based NMS 204 ).
- operating system services 220 e.g., Linux system services
- web service 222 e.g., web service 222 , deep packet inspection (DPI) service 224 , application configuration 226 , NMS events interface 228 , operating system calls 230 , MASE application programing interface 232 (MASE API), MASE application 210 (downloaded from cloud-based NMS 204 ).
- DPI deep packet inspection
- AP device 202 also includes physical resources on the AP device such as a network processor, a WLAN chipset capable of connecting more than 50 wireless clients to a wired network at the same time, an AP RAM memory, permanent storage, AP multi-core central processing units (CPU), and an AP file system.
- physical resources are on the AP device.
- Application marketplace 258 is a repository of third party applications that are developed using the MASE technology, according to certain embodiments.
- Third party application developers use the MASE software development kit (SDK) to develop MASE applications that can be deployed in a given AP device.
- SDK software development kit
- the third party application developers can define the type of MASE APs where the applications can be deployed.
- the third party application developers can define their applications as private or public applications. Private applications can only be deployed at APs that are managed by the same AP providers that the third party developers are associated with.
- Public applications can be deployed at any of the APs, and a licensing scheme is available for the third party developer to collect licensing fees from AP owners that deploy the public applications, according to certain embodiments.
- third party application developers can define the type of configuration that can be applied to the applications, such as the name of the configuration parameters, the type of the parameters and the range or possible values of the parameters.
- a configuration can be defined for a group of APs where the application is deployed, or a configuration can be defined for a specific AP.
- the MASE SDK includes the open source OpenWRT build environment, with which the third party developer can use to build the application and package the application in the standard ipk package.
- the SDK also includes the MASE API library and the header file, which can be included in the application if the third party developer decides to leverage the system services provided by the MASE API.
- the SDK includes a suite of tools that allow the third party developer to manage the installation of the applications of the third party developer on a MASE AP device as part of the development test bed.
- the MASE API is required to run in a developer mode, according to certain embodiments.
- the AP handles the installation commands generated by the tool suite.
- AP owners can select MASE applications to be deployed on their APs. For example, AP owners can select either private applications that are developed by their own developers, or public applications that are developed by any third party developers. When the applications are selected to be deployed or installed on some APs, configurations can be specified for the group of APs based on the requirements of the particular deployments. Further, the AP owners can decide whether the applications, after they are installed on the APs, should start or stop running, or whether they should be restarted. The AP owners can do so by specifying the running status of the applications.
- the cloud-based NMS 204 implements different interfaces and tools for application maintenance. For example, the status and health of the MASE applications (as well as memory usage, CPU usage and disk usage) can be monitored at cloud-based NMS 204 . The application status and health are updated consistently.
- MASE applications 210 running in containers on the AP have access to standard Linux system services 220 such as the file system and internetworking features (for example, TCPIP protocol, UDP protocol, HTTP server connections, and various software libraries).
- standard Linux system services 220 such as the file system and internetworking features (for example, TCPIP protocol, UDP protocol, HTTP server connections, and various software libraries).
- advanced features such as Web services are available to the MASE applications via MASE API.
- the MASE API 232 supports cloud-based application configuration and event logging.
- the embodiments are not limited to Linux operating systems. The type of operating systems may vary from implementation to implementation.
- Linux system services 220 include but are not limited to: 1) Pre-allocated disk space for a variety purposes such as application data storage, 2) IP internetworking through UNIX socket API, and 3) Standard UNIX syslog service.
- MASE applications 210 are installed and run within the confinement of container 208 .
- Container 208 is constructed based on Linux technologies, according to certain embodiments. For example, container 208 uses the same standard Linux tools and libraries built in the AP image from the OpenWRT distribution.
- Each MASE application 210 has pre-allocated disk space in an isolated file system inside the container.
- Each MASE application 210 has full read and write access to its dedicated pre-allocated disk space. However, the MASE application 210 has only read access outside its dedicated pre-allocated disk space within the container when running as a non-privileged user.
- the construction of container 208 may be based on other technologies and is not limited to Linux technologies and may vary from implementation to implementation.
- MASE applications 210 are packaged using the standard ipk package format that is used by the open source OpenWRT build environment.
- Application installation at a given AP is managed through the cloud-based NMS 204 remotely.
- Standard application installation scripts e.g., preinst, postinst, prerm, postrm
- applications are installed or uninstalled at a given AP, according to certain embodiments.
- the running status of a MASE application 210 is managed remotely at cloud-based NMS 204 .
- the MASE application 210 is started or stopped using a service status management mechanism via application “init” script that is installed at location /etc/init.d/, for example.
- Init script commands e.g., start, stop, restart, status, are used to manage the application status.
- MASE applications 210 while MASE applications 210 are running within the confinement of a container 208 on the AP, the MASE applications 210 have full access to other system resources on the AP such as the physical memory and CPU cycles of the AP.
- a given MASE application 210 can specify the amount of system resources that the given MASE application needs in the application package manifest.
- the amount of system resources are capped at a hard limit when the MASE application 210 is installed on the one or more APs.
- MASE applications 210 are more or less continuously monitored and can be restarted as soon as they are found to have used up too many AP resources based on the predefined limits, according to certain embodiments.
- application configuration is managed at cloud-based NMS 204 .
- the configuration information 226 is eventually passed to the MASE application 210 via the MASE API 232 . If the MASE application 210 requires configuration support from cloud-based NMS 204 , the MASE application 210 can use the MASE API 232 in order to receive the application configuration information.
- Web applications are server side applications that implement specific Web requests from wireless clients. According to certain embodiments, there are three models of web applications for accessing the web requests. According to certain embodiments, the three models of web applications for accessing the web requests are: 1) Proxy service model based on domain names specified by applications, 2) Static web content model, 3) FCGI applications model.
- MASE API 232 provides advanced HTTP proxy service based on proxy requirements specified by the MASE application 210 . When HTTP proxy service is enabled for certain domain names, the HTTP requests are proxied and forwarded to proxy HTTP/TCP ports that the MASE application 210 specifies. The MASE application 210 can implement an HTTP server on such proxy HTTP/TCP ports to service the proxied HTTP requests received at those HTTP/TCP ports.
- Web service 222 can host static Web content packaged in the application ipk package.
- Third party developers can package Web content, such as HTML pages, JavaScript programs, and multimedia files (pictures/audio/video) in the MASE application package together with a Web hosting manifest that specifies the base URL locations (including host names and base paths) for the Web content.
- Web service 222 is reconfigured to support the URL locations specified in the Web hosting manifest.
- third party applications can implement the standard fast common gateway interface (FCGI).
- FCGI applications are installed in the same Linux container environment as regular third party MASE applications on the APs, but are invoked as FCGI applications with Web service 222 handling the Web request before invoking the third party applications via the standard FCGI protocol.
- Web service 222 can be configured to invoke the third party applications for selected URLs based on FCGI Web hosting manifest, which is packaged together with the third party FCGI application in the application ipk package. For example, when a client device requests a web page or a video from the internet, web service 222 will check to see if there is a match in the URL list. If there is a match, web service 222 invokes the FCGI application, which in turn, will analyze the request from the client device and decide on the appropriate response to send to the client device.
- DPI service 224 supports DPI related applications by providing advanced flow-based packet sniffing. Packets can be captured on a per flow basis, with adjustments allowed to sniff more or fewer packets for a particular flow.
- DPI analytic applications can use the MASE API 232 to export statistics and reports to the cloud-based NMS 204 .
- Cloud-based NMS 204 can provide cloud-based storage and GUI integration for display purposes.
- MASE API 232 allows the third party application to access various system services. For example, application configuration is managed at cloud-based NMS 204 , which sends the configuration information to the relevant AP where the application is deployed. The configuration information is sent to the application via MASE API 232 . As another example, for all the HTTP requests that the MASE application 210 would like Web service 222 to proxy on behalf of the MASE application 210 , the MASE application 210 can specify the domain names of such URLs via the MASE API 232 together with the proxy HTTP/TCP port number. The MASE API 232 reconfigures the networking stack on the AP to proxy the HTTP requests destined for such domain names, and forwards the requests to the specified proxy HTTP/TCP ports.
- the third party application can implement a Web server on such proxy HTTP/TCP ports to receive the proxied HTTP requests. Further, flow-based packet sniffing is supported to capture packets for DPI analysis. Statistics and reports are exported to cloud-based NMS 204 for integration with cloud-based services provided by cloud-based NMS 204 . In addition, cloud-based NMS 204 implements an event log to log various kinds of system events related to activities at the APs. MASE applications 210 can generate events using MASE API 232 so that such events can be logged at the event log in cloud-based NMS 204 .
- FIGS. 3A, 3B and 3C illustrate management of MASE applications, according to certain embodiments.
- FIG. 3A shows MASE application environment 300 A on an AP.
- FIG. 3A shows container 302 that includes an application manager agent 304 , and MASE application 308 .
- Application manager agent 304 communicates with the AP's application manager daemon 312 via inter-process communication (IPC) 316 .
- IPC inter-process communication
- application manager agent 304 monitors CPU usage, memory usage (e.g., RAM usage), and disk usage of the MASE application 308 .
- Application manager daemon 312 handles problems locally at the AP based on pre-determined policies, for example.
- Application manager agent 304 communicates information 322 (such as application running status, health status) to application manager daemon 312 .
- Application manager agent 304 also communicates relevant information with MASE application 308 .
- MASE application 308 uses MASE API 310 to communicate information 320 (e.g., application events) to application manager daemon 312 via MASE manager API 314 .
- MASE application 308 receives application configuration information 318 from application manager daemon 312 via MASE manager API 314 and MASE API 310 .
- application manager daemon 312 communicates event information 326 to the monitor daemon 328 on the AP.
- application manager daemon 312 can receive information 324 (e.g., installation configuration information, application status configuration information, application configuration information) from the cloud-based NMS, according to certain embodiments.
- FIG. 3B shows a different embodiment of application manager agent 304 b.
- Application manager agent 304 b includes a MASE DPI API 307 that can receive information such as application configuration and application status configuration ( 318 , 318 b ) from MASE manager DPI API 314 b. Further, application manager agent 304 b uses API 309 to communicate (e.g., DPI information) with MASE application 308 .
- MASE application 308 uses MASE DPI API 310 b to send buffer file upload 320 b to application manager daemon 312 through MASE manager DPI API 314 b.
- FIG. 3C shows a different embodiment of MASE application 308 c, which uses MASE API 310 to communicate transparent proxy domain information 320 c to application manager daemon 312 via MASE manager API 314 .
- FIG. 4 illustrates MASE libraries to support MASE applications installed in a given AP device, according to certain embodiments.
- FIG. 4 shows application manager library layers 402 and Application/application manager agent library layers 404 in a MASE library architecture 400 , according to certain embodiments.
- Application manager library layers 402 are not exposed to third party developers and comprise a private dpi library layer 406 , and a private API library layer 412 .
- Application/application manager agent library layers 404 comprise a receive-only DPI library layer 408 a send-only DPI library layer 410 , and an API library layer 414 , according to certain embodiments.
- FIG. 4 also shows a communication transport layer 416 for IPC messages, according to certain embodiments.
- FIG. 5 illustrates segregation of a portion of the AP file system to confine MASE applications, according to certain embodiments.
- FIG. 5 shows the file system 500 of a given AP.
- file system 500 comprises: 1) selected directories 502 that can be shared with the MASE application container, via hard links 520 , 2) proprietary directories 504 that are not shared with the MASE application container, and 3) the container file system 506 .
- Container file system 506 uses change root (chroot) for all guest processes (directories 508 , and 510 ).
- the proprietary directories 504 include a virtual disk partition in the form of a disk image that contains the writeable disk space to which a given MASE application container can write data.
- Mount 522 is a Linux process for making the virtual disk partition accessible by the MASE applications through regular Linux filesystem management commands (e.g., commands related to creating files, writing to a file, creating subdirectories, etc.).
- regular Linux filesystem management commands e.g., commands related to creating files, writing to a file, creating subdirectories, etc.
- the MASE applications on the AP device has access to such a virtual disk partition.
- the embodiments are not limited to Linux-type file systems. The file systems may vary from implementation to implementation.
- a method of WiFi networking comprises: deploying containerized applications (MASE applications) on a wireless access point device of a plurality of wireless access point devices that are associated with a WiFi network for connecting a plurality of wireless client devices to a wired network.
- the method comprises using at least one application container on the AP device.
- the method further comprises installing one or more third party applications in the at least one application container on the AP device.
- the method further comprising using an AP device that includes: 1) a WLAN chipset on the AP device, which WLAN chipset is capable of connecting at least about 50 wireless client devices at the same time to the wired network that is associated with the AP device (preferably, the WLAN chipset is capable of connecting at least 100 wireless client devices to the wired network that is associated with the AP device), 2) a stand-alone multi-core CPU (central processing unit) on the AP device, which stand-alone multi-core CPU is capable of hosting relevant WiFi related connection services for wireless client devices, as well as capable of hosting/supporting MASE applications, 3) a high-speed and large RAM memory on the AP device, which RAM memory is capable of hosting MASE applications on the AP device, 4) high-speed and large permanent storage on the AP device for supporting MASE applications and for storing large amounts of cached data to support better web-services delivery, 5) dedicated network processor on the AP device, which dedicated network processor is capable of analyzing and controlling IP data packets and for controlling data traffic to achieve reasonable
- the method further comprises using a pre-allocated segregated disk space within a file system on the AP device that can be accessed and used by the containerized applications that are installed on the AP device.
- the method further comprises using an application run-time management system on the AP device to monitor the execution of the containerized third party applications and including monitoring the CPU usage, RAM usage, disk storage usage, application running status, application health status, and application events.
- the method further comprises using a MASE application programming interface (MASE API) on the AP device to provide web services to the MASE applications installed on the AP device and to provide deep packet inspection services to the MASE applications installed on the AP device.
- MASE API MASE application programming interface
- the consumption of airtime allocated to each connected wireless client device (also referred to herein as a connected wireless node) of at least a subset of wireless client devices/nodes that are connected to corresponding access point devices in a wireless system is distributed across a cycle time.
- a cycle time is a predetermined value and is the duration of time per iteration.
- the consumption of airtime that is allocated to each wireless client device/node is distributed across a cycle period based on calculating the airtime required for the given wireless client device/node of a plurality of wireless client devices/nodes for transmitting a packet when connected to a corresponding access point device in the wireless communication system.
- the airtime allocated to each connected wireless client device/node is estimated based on comparing the required airtime for the respective connected wireless client device/node with the available airtime for each connected wireless client device/node, where available time is the total available airtime in the cycle divided by the total number of connected wireless nodes.
- the consumption of airtime that is allocated to each wireless client device is distributed across a cycle period based on comparing the consumed airtime of the given wireless client device/node with the elapsed cycle time when the given wireless client is connected to a corresponding access point device in the wireless communication system.
- the distribution of consumption of airtime allocated to each connected wireless client device/node across a cycle period in the wireless communication system of a plurality of wireless client devices/nodes includes throttling transmission of packets of the given connected wireless client device/node based on one or more pre-determined criteria.
- the distribution of consumption of airtime allocated to each connected wireless client device/node across a cycle period in the wireless communication system of a plurality of wireless client devices/nodes includes providing feedback information including computation of airtime transmission of a given packet by the given connected wireless client device/node through its associated access point device.
- a given connected wireless client device/node in a wireless communication system can consume its corresponding allocated airtime per cycle all at once in the cycle. Such a connected wireless client device/node can result in hogging the transmission channel at its associated access point device.
- a given connected wireless client device/node in a wireless communication system is controlled to consume its allocated airtime in a controlled distributed manner across a cycle to prevent excessive hogging of a transmission channel and/or to prevent excessive dropping of packets.
- such a wireless communication system periodically estimates the airtime required by a given connected wireless node and throttles the given node's transmission of packets across a given cycle based on the usage of airtime per cycle. Further, such a wireless communication system provides correction feedback information per cycle to the airtime estimation engine.
- FIG. 6 is a high-level network diagram showing aspects of distributing the consumption of airtime allocated to each connected wireless client device across a cycle time in a wireless communication network, according to certain embodiments.
- the airtime estimation engine calculates the node allocated airtime for a given connected wireless node of at least a subset of a plurality of wireless nodes connected to a corresponding access point in the wireless communication network to produce the corresponding node allocated airtime result at block 604 .
- Node allocated airtime is the duration of time allocated to a connected wireless node for receiving and transmitting packets per cycle time, according to certain embodiments.
- an airtime distribution engine receives an elapsed cycle time percentage value as an input from block 606 and also receives the node allocated airtime result from block 604 and uses such inputs for determining whether to send a packet that the given connected wireless node is attempting to send (or receive) to a throttler at block 610 or to send the packet to the wireless transceiver at block 612 .
- FIG. 6 also shows some functional aspect of the wireless transceiver at the physical layer shown at block 614 .
- the wireless transceiver adds the time that the given connected wireless node used to transmit ( 616 ) a given packet with the time the given connected wireless node used to receive ( 618 ) a packet and send the information to a feedback engine 620 .
- the feedback engine 620 updates a feedback table 622 .
- the feedback table includes information such as number of transmission retries, data rate, unused airtime, drop airtime, transmission airtime, receiving airtime, Request-to-Send (RTS) flags, Clear-to-Send (CTS) flags etc, according to certain embodiments.
- the feedback engine send information on the airtime consumed by the given connected wireless node to the airtime estimation engine. The process described with reference to FIG. 6 is repeated for each connected wireless node of at least a subset of a plurality of wireless nodes connected to the corresponding access point in the wireless communication network, according to certain embodiments.
- FIGS. 7A, 7B, 7C, and 7D are high-level flow charts showing aspects of an airtime calculation engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- the processes in FIGS. 7A, 7B, 7C, and 7D are described with respect to a given connected wireless node and are repeated for each connected wireless node of at least a subset of the plurality of wireless nodes connected to the corresponding access point in the wireless communication network, according to certain embodiments.
- the airtime estimation engine calculates the total available airtime per cycle for a given connected wireless node.
- the channel congestion is the amount of channel capacity that is currently in use and therefore not available for transceiving packets for the given connected wireless node.
- total free airtime total available airtime.
- the airtime estimation engine determines the total number of wireless nodes connected to the given access point (referred herein as “connected wireless nodes”) in the wireless communication network.
- the airtime estimation engine calculates the airtime than can be shared by the connected wireless nodes.
- Node initial airtime total available airtime/number of connected wireless nodes, according to certain embodiments.
- the airtime estimation engine calculates the airtime used by the given connected wireless node (also referred to as “node used airtime” or “node consumed airtime”) in the previous cycle.
- the node consumed airtime is provided by the feedback engine as explained in greater detail herein.
- the node used airtime will be zero for wireless nodes that are newly connected to the access point in the current cycle.
- Re-transmissions are also known as retries.
- a WiFi medium is not a deterministic medium because of channel interference. There may be many retries in the channel due to interference. Retries are the number of times a connected wireless node tries to transmit a data packet until the packet is successfully transmitted. For example, a successful transmission is when the connected wireless node receives an acknowledgment for each of the data packets it has transmitted. A given connected wireless node will retry sending packets that are not acknowledged by the receiver. Retries are more frequent when the WiFi channel is congested, or if the given connected wireless node is far away from the access point and/or is not in the line of sight of the access point. In some wireless chipsets of wireless nodes, the number of retries may be pre-programmed for every packet so that the packets are resent automatically.
- RTS/CTS Request to Send/Clear to Send
- RTS/CTS Request to Send/Clear to Send
- the amount of airtime consumed by RTS/CTS messages needs to be accounted for with respect to the given connected wireless node that is attempting to send the data packet. For purposes of illustration, assume that connected Wireless Node 1 and Wireless Node 2 are attempting to send packets. Further assume that Wireless Node 1 is sending packets with RTS/CTS exchange mechanism. The airtime consumed by Wireless Node 1 will be higher than that of Wireless Node 2 .
- the airtime estimation engine calculates the error correction based on the inputs for the previous cycle.
- the error correction is needed to calculate the airtime required by the given connected wireless node for transmitting/receiving packets.
- the airtime estimation engine calculates the airtime required by the given node in the current cycle to transmit/receive packets.
- the node required airtime previous cycle node allocated airtime+error correction. Control of the process passes to block A 1 of FIG. 7B .
- the airtime estimation engine determines if there are more connected wireless nodes for which airtime needs to be estimated. If there are more connected wireless nodes for which airtime needs to be estimated, then control passes to block A 2 of FIG. 7A . From block A 2 of FIG. 7A , control passes back block 704 that is previously described herein with reference to FIG. 7A . If there are no more connected wireless nodes for which airtime needs to be estimated, then control passes to block B of FIG. 7C as described herein.
- the node allocated airtime is set as equal to the node required airtime, according to certain embodiments.
- the node minimum airtime threshold is a predetermined value, according to certain embodiments. If it is determined that the node allocated airtime is not less than the node minimum airtime threshold, then control passes to block 730 . Block 730 is previously described above. If it is determined that the node allocated airtime is less than the node minimum airtime threshold, then at block 728 , the node allocated airtime is set as equal to node minimum airtime threshold, in order to avoid a zero value, according to certain embodiments.
- the purpose of a “node share” is to share the unused airtime among connected wireless nodes that need more airtime.
- control passes to block B 1 of FIG. 7D . If at block 746 , it is determined that the node required airtime is not greater than the node allocated airtime, then control passes back to block 744 where the airtime estimation moves to the next connected wireless node for consideration as described with reference to block 746 .
- control passed to block 760 where the airtime estimation engine calculates the remaining airtime for the given connected wireless node is calculated as: remaining airtime remaining airtime ⁇ node share.
- FIG. 8 is a high-level network flow chart showing aspects of an airtime distribution engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- the airtime distribution engine finds the destination wireless node at block 806 .
- the airtime consumed by the destination node for transmitting packets, the airtime consumed by the destination node for receiving packets, and the airtime consumed by the destination node due to re-transmissions information can be obtained from the feedback table.
- the airtime distribution engine determines if the elapsed cycle time percentage minus the destination node consumed airtime percentage is greater than the throttle threshold value.
- the throttle threshold value is a predetermined value according to certain embodiments. If it is determined that the elapsed cycle time percentage minus the destination node consumed airtime percentage is not greater than the throttle threshold value, then at block 818 , the given data packet is queued to the throttler. However, if it is determined that the elapsed cycle time percentage minus the destination node consumed airtime percentage is greater than the throttle threshold value, then at block 816 , the airtime distribution engine calculates the required airtime for transmitting the given data packet (“required packet airtime”) based on information for the feedback table.
- the required packet airtime is a function of packet length, selected data rate, previous retries for the selected data packet rate obtained from the feedback table.
- the airtime distribution engine determines if the required airtime for transmitting the given data packet is greater than the destination node's remaining airtime. If it is determined that the required airtime for transmitting the given data packet is greater than the destination node's remaining airtime, then control passes to block 818 where the given data packet is queued to the throttler. However, if it is determined that the required airtime for transmitting the given data packet is not greater than the destination node's remaining airtime, then the data packet is sent to the wireless transceiver for transmission. The airtime distribution process ends at block 824 .
- FIG. 9 is a high-level network flow chart showing aspects of a throttler for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments.
- block 902 operates on the next packet (current packet) in the throttler queue.
- the throttler determines if the continuous dropped packet count of a given node associated with the current packet is greater than the dropped packet count threshold.
- the dropped packet count threshold is a predetermined value. If it is determined that the continuous dropped packet count of the given node associated with the current packet is greater than the dropped packet count threshold, then the packet is sent to the wireless transceiver at block 906 .
- the given current packet is an aggregate packet. If it is determined that the given current packet is an aggregate packet, then at block 918 , the size of the aggregate packet is limited by consulting the feedback table at block 920 . At block 922 , it is determined if the size of the aggregate packet is greater than the minimum required aggregate size. If it is determined that the size of the aggregate packet is not greater than the minimum required aggregate size, then at block 924 , the packet is dropped.
- the aggregate packet is sent to the wireless transceiver. However, if it is determined at block 926 that the transmission time for the aggregate packet is not less than the remaining airtime of the destination node, then at block 928 , the aggregate packet is dropped.
- the packet is a UDP packet. If it is determined that the packet is a UDP packet, then at block 916 , the UDP packet is queued for transmission during the idle cycle time if there is an un-utilized airtime available at the end of cycle. According to certain embodiments, a decision may be made to drop the UDP packet if the number of packets queued thus far for the given connected wireless node exceeds a predetermined maximum value. When queuing each of the packets, the throttler will track the time required to transmit each of the packets based on the size of the packet, according to certain embodiments.
- the throttler will transmit the queued UDP packets until one of the connected wireless nodes makes a transmission request. At the end of the cycle period, the packets remaining in the UDP queue will be dropped, according to certain embodiments.
- a block 912 it is determined that the packet is not a UDP packet (e.g., it is a TCP packet), then at block 914 , the packet is dropped. The process ends at block 908 .
- the feedback engine provides feedback to the airtime estimation engine.
- the feedback information can include the number of packet dropped, the time take for transmission, the time wasted in retries and the time consumed for receiving the packets.
- the feedback engine can be queried by the airtime distribution engine for the required airtime to transmit a packet.
- the airtime of a packet is computed at the completion of transmission of each packet.
- the following information can be obtained from the access point hardware for computing airtime:
- the access point hardware decides the rate of transmission for packets associated with each wireless node.
- the feedback engine builds a lookup table referred to as a feedback table.
- the feedback table includes supported PHY layer statistics such as:
- FIG. 10 is a high-level network flow chart showing aspects of a feedback engine for wireless client devices/nodes connected to wireless access point devices in a wireless network, according to certain embodiments.
- the feedback engine obtains the rate of transmission information of the transmitted packet from the access point hardware.
- the feedback engine obtains the number of retries associated with the transmission rate information obtained at block 1002 .
- the feedback engine obtains the packet transmit airtime from the access point hardware.
- the node transmitted airtime is updated by the feedback engine.
- the feedback table is updated by the feedback engine. The process ends at block 1012 .
- a wireless communication system comprises an airtime estimation engine for determining a respective node allocated airtime in a cycle time period for a given wireless node connected to an access point in the wireless communication system and is based on comparing a required airtime for the given wireless node with an available airtime for the given wireless node in the wireless communication system.
- the available time is a total available airtime in a cycle divided by a total number of connected wireless nodes in the wireless communication system.
- such a system further comprises an airtime distribution engine for controlling the given wireless node's consumption of its node allocated airtime across the cycle time period and is based on comparing a consumed airtime percentage of the given wireless node with an elapsed cycle time percentage of the cycle time period.
- the wireless communication system further comprises a throttler engine for determining a packet type of each respective packet in a queue of the throttler engine.
- the throttler engine queues the respective packet for transmission during idle cycle time if the packet type is determined to be a UDP type.
- the throttler engine limits packet size if the packet type of the respective packet is determined to be an aggregate type. According to certain embodiments, the throttler engine drops the respective packet if the packet type is determined to be a TCP type.
- the airtime distribution engine calculates a required packet airtime for a respective packet based on information from a feedback table. According to certain embodiments, the feedback engine obtains a respective packet transmission time.
- the wireless communication system further comprises a feedback engine for obtaining a respective packet transmission rate.
- the feedback engine obtains a respective packet transmission time.
- the feedback engine updates a feedback table.
- load balancing across access points in the wireless environment may be achieved using a cloud based, controller-less, distributed solution, according to certain embodiments.
- a distributed solution of load balancing across access points has a faster convergence rate than non-distributed solutions.
- a distributed solution of load balancing across access points obviates a single point of failure.
- a distributed solution of load balancing across access points reacts more efficiently to changes in the wireless environment.
- a distributed solution of load balancing across access points in a cloud based, controller-less wireless environment helps reduce capital expenditure, operational expenditure and reduces complexity of the wireless environment.
- a distributed solution of load balancing includes using at least of subset of the following:
- the client when a client would like to connect to an access point, the client (also referred to as a “requesting client”) sends, to access points that are nearby, a probe request to connect.
- the client also referred to as a “requesting client”
- each access point that receives a probe request from the requesting client obtains information on the best access point for connecting to the requesting client.
- an access point decides to respond to the requesting client based on at least the information on the best access point for connecting to the requesting client. According to certain embodiments, an access point decides to respond to the requesting client based on one or more criteria from a set of predetermined criteria. According to certain embodiments, examples of the predetermined criteria can include but are not limited to: a channel congestion weighting factor, a number of connected clients weighting factor, a CPU weighting factor, an acceptance weight threshold.
- a load balancer maintains a list (that is associated with a given requesting client) of access points for ranking (also referred to as a “ranking list’).
- the load balancer ranks the ranking list of access points as a function of a channel congestion, a CPU usage and number of connected clients of a given AP.
- the load balancer ranks APs in the ranking list in view of each AP's deviation from the worst case value for each parameter of a predetermined set of parameters.
- FIG. 11 is a high-level network diagram showing aspects of a distributed load balancing solution, according to certain embodiments.
- a client 1102 also referred to as “a requesting client” that wishes to connect to the best access point (or connect to a suitable access point) sends a request probe to each wireless access point that is nearby the requesting client, according to certain embodiments.
- Each access point that receives a probe request is also referred to as a “current AP”.
- current AP 1104 communicates with a client load balance server 1106 . Certain aspects of current AP 1104 are described in greater detail with reference to at least FIG. 12 , herein.
- client load balance server 1106 determines ( 1108 ) the distance of the current AP 1104 from requesting client 1102 based on the received signal strength indication (RSSI) of requesting client 1102 as perceived by current AP 1104 .
- RSSI received signal strength indication
- client load balance server 1106 maintains a group of APs for each requesting client and determines if the current AP 1104 is to be added ( 1110 ), based on the RSSI of the requesting client 1102 , to a “ranking list” associated with requesting client 1102 . Certain aspects of the client load balance server 1106 are described in greater detail with reference to at least FIG. 13 , herein.
- client load balance server 1106 ranks ( 1112 ) the APs in the ranking list associated with requesting client 1102 to determine the best AP (“BAP”). Certain aspects of the ranking process are described in greater detail with reference to at least FIG. 13 , FIG. 14 and FIG. 15 , herein.
- client load balance server 1106 communicates ( 1114 ) information on a set of parameters of the BAP to current AP 1104 .
- a decision logic engine decides ( 1116 ) whether to send a response ( 1118 ) to the requesting client 1102 or not to send a response ( 1120 ) to the requesting client 1102 . Certain aspects of such a decision process are described in greater detail with reference to at least FIG. 16 , and FIG. 17 , herein.
- the process described with reference to FIG. 11 occurs for each of the nearby APs that receive the request probe from the requesting client 1102 .
- FIG. 12 is a high-level flow chart that illustrates some of the functions of a current AP that receives a probe request, according to certain embodiments.
- the current AP receives a probe request from a requesting client.
- the current AP determines if the requesting client is a new client. If the current AP determines that the requesting client is a new client, then at block 1206 , the current AP sends a request to client load balance server (also referred to as a “CLS”) to find the best access point (BAP) for the requesting client.
- client load balance server also referred to as a “CLS”
- the current AP along with the request to find BAP, the current AP sends at least a subset of the following information to the CLS: 1) RSSI of the requesting client as seen by the current AP, 2) current channel congestion (or channel utilization, “CU”) associated with the current AP, 3) current CPU utilization of the current AP, 4) the number of clients connected to the current AP, 5) media access control address (MAC address) of the current AP, and 6) media access control address (MAC address) of the requesting client.
- the current AP waits for a response from the CLS for up to a predetermined maximum silent period, according to certain embodiments.
- the CLS determines the identity of the BAP and related information (BAP parameters) in response to the current AP's request referred to at block 1206 .
- the determination of the identity of the BAP and related BAP parameters is described in greater detail with reference to at least FIGS. 13 and 14 , herein.
- the current AP sends a response to the requesting client. However, if the time for receiving a response from the CLS is less than the predetermined maximum silent period, then at block 1212 , the current AP waits for the information on the BAP and sends such information to a decision logic engine 1218 .
- the decision logic engine may be physically part of the current AP device. According to certain other embodiments, the decision logic engine may be remote from the current AP device. The manner in which the decision logic engine decides whether to respond to the requesting client is described in greater detail with respect to at least FIG. 16 and FIG. 17 , herein.
- the current AP receives a decision from the decision logic engine and determines whether the decision logic engine has decided to accept or reject the requesting client. If it is determined that the requesting client should be accepted, the current AP sends a response to the requesting client at block 1222 . If it is determined that the requesting client is to be rejected, the current AP will not respond to the requesting client, according to certain embodiments.
- the current AP determines if at block 1204 , the current AP determines that the requesting client is not a new client, then at block 1224 , the current AP determines if the previous decision made by the decision logic engine is still valid based on how long ago the decision was made. For example, the previous decision remains valid if the age of the previous decision is less than a maximum decision age value.
- the current AP determines if the previous decision made by the decision logic engine is not valid. If at block 1224 the current AP determines that the previous decision made by the decision logic engine is not valid, then at block 1226 , the current AP determines if the BAP is a known BAP. If the current AP determines that the BAP is not known, then control passes to block 1206 and block 1206 has been previously described above.
- the current AP determines if the age of the BAP is less than a maximum age threshold. If the current AP determines that the age of the BAP is less than a maximum age threshold, then control passes to block 1218 so that the decision logic engine can determine whether to accept or reject the requesting client as previously described above. If however, the current AP determines that the age of the BAP is greater than the maximum age threshold, then control passes to block 1206 and block 1206 has been previously described above.
- FIG. 13 is a high-level flow chart that illustrates some of the operations of the client load balance server (CLS).
- CLS client load balance server
- the CLS receives, from the current AP, the request to find the best access point (BAP) for the requesting client, as previously described with reference to at least FIG. 12 above.
- the CLS along with the request to find the BAP, receives the following information from the current AP: 1) RSSI of the requesting client as seen by the current AP, 2) current channel congestion (or channel utilization percentage, “CU”) associated with the current AP, 3) current CPU utilization percentage of the current AP, 4) the number of clients connected to the current AP, 5) media access control address (MAC address) of the current AP, and 6) media access control address (MAC address) of the requesting client.
- RSSI of the requesting client as seen by the current AP
- CU channel utilization percentage
- the CLS associates a group of APs with each client that is known to the CLS.
- the CLS receives, from the current AP, the request to find the best access point (BAP) for the requesting client
- the CLS adds, at block 1304 , the current AP to the group of APs associated with the requesting client if the requesting client is previously known to the CLS.
- the CLS creates a new group of APs for the requesting client, according to certain embodiments.
- the CLS determines the distance of the requesting client from the current AP.
- the distance can be determined using triangulation techniques.
- the CLS determines if the distance of the requesting client from the current AP is less than a predetermined distance threshold. If the distance is less than the predetermined distance threshold, then at block 1312 the CLS adds the current AP to a ranking list associated with the requesting client. However, if the distance is not less than the predetermined distance threshold, then at block 1310 , the CLS omits the current AP from the ranking list associated with the requesting client, according to certain embodiments.
- the CLS determines if the number of APs in the ranking list is greater than 1. If the number of APs in the ranking list is greater than 1, then at block 1316 , the CLS ranks the AP in the list. The manner of ranking is described in greater detail with reference to at least FIG. 14 and FIG. 15 , herein, according to certain embodiments. Such a ranking identifies the best AP (BAP).
- the CLS sends information of the BAP, such as the identity of the BAP and other BAP parameters, to the current AP.
- the BAP information includes at least a subset of: 1) RSSI of the requesting client as seen by BAP, 2) channel utilization percentage associated with BAP, 3) BAP CPU utilization percentage, 4) the number of clients connected to BAP, 5) media access control address (MAC address) of BAP, and 6) media access control address (MAC address) of the requesting client.
- the CLS determines that the number of APs in the ranking list is not greater than 1, then at block 1320 , the CLS waits for more APs to make request for BAPs. According to certain embodiments, the CLS waits for a period up to a predetermined maximum silent period.
- the CLS determines if the predetermined maximum silent period has expired. If the predetermined maximum silent period has expired then at block 1324 , the CLS designates the current AP as BAP and sends the BAP information to the current AP at block 1318 as previously described.
- FIG. 14 is a high-level flow chart that illustrates some of the features of the ranking operation for ranking the APs, according to certain embodiments. The higher the AP's rank, the better the AP.
- the CLS considers each AP in the ranking list in view of each AP's deviation from the worst case value for each parameter of a predetermined set of parameters, as described with reference to blocks 1404 , 1406 , 1408 of FIG. 14 .
- the CLS determines the absolute value of the difference between channel utilization maximum threshold value and current AP channel utilization.
- the CLS determines the absolute value of the difference between CPU usage max threshold value and current AP CPU usage adds it to the result of block 1404 .
- the CLS determines the absolute value of the difference between number of clients connected to the best AP and the number clients connected to the current AP and adds it to the result of block 1406 .
- the result of block 1408 is the rank of the respective current AP.
- the CLS determines if the current AP has a rank that is greater than the rank of the best AP thus far. If the CLS determines that the current AP's rank is greater than the best rank, then at block 1414 , the CLS designates the current AP as the BAP. Next control passes to block 1412 which is described below.
- the CLS determines if there are more APs in the rank list. If there are no more APs in the rank list, then at block 1416 , the CLS sends the BAP information to the current AP. If there are more APs in the rank list then control passes back to block 1402 to determine the rank of the next current AP.
- the APs in the ranking list are ranked as follows based on empirical analysis:
- FIG. 15 is a graph that illustrates aspects of the ranking logic used for load balancing clients across APs in a wireless environment, according to certain embodiments.
- Ranking logic graph 1500 of FIG. 15 shows Rank 1502 as the vertical axis and Rank vs channel utilization/CPU utilization (CU/CPU) 1504 as the horizontal axis.
- Graph 1500 also shows the rank vs CPU plot 1506 and the rank vs CU plot 1508 .
- Graph 1500 also shows the worst case value 1510 based on the example above.
- the current APs are ranked based on the respective current AP's deviation from the worst case value for each parameter of a predetermined set of parameters, according to certain embodiments.
- the parameters include channel utilization, CPU utilization and number of connected clients, according to certain embodiments.
- FIG. 16 is a high-level flow chart that illustrates some aspects of the decision logic engine used by the current AP (when the current AP receives the BAP information from the CLS) to decide whether to respond or ignore the requesting client that is requesting connection with the current AP.
- such a decision of the current AP can be any one of the following types: 1) aggressive, 2) inclined, and 3) fair.
- the current AP responds immediately to the requesting client without further delay or further processing.
- the current AP processes certain parameters as described in greater detail below.
- a fair decision is selected, the current AP will attempt to divide the number of clients fairly across the current AP and the BAP.
- the decision logic engine determines a channel utilization (CU) weight between BAP and the current AP.
- the decision logic engine determines if the weight determined at block 1602 is greater than a predetermined maximum aggressive weight threshold value. If the weight determined at block 1602 is greater than the maximum aggressive weight threshold value then, at block 1608 , the decision logic engine determines if the weight determined at block 1602 is greater than or equal to a predetermined “accept weight” threshold value. If the weight determined at block 1602 is greater than or equal to the predetermined “accept” threshold value, then at block 1610 the decision status is set to “accept” (so that the current AP can send a response to the requesting client to allow the requesting client to connect to the current AP. However, if the weight determined at block 1602 is not greater than or equal to the predetermined “accept” threshold value, then at block 1612 the decision status is set to “reject” (so that the current AP can ignore the requesting client probe request).
- CU channel utilization
- the decision logic engine determines that the weight determined at block 1602 is not greater than the predetermined maximum aggressive weight threshold value, then at block 1606 , the decision logic engine determines the CPU weight between the BAP and the current AP.
- the decision logic engine adds the CPU weight to the CU weight, according to certain embodiments.
- the decision logic engine determines if the combined CPU and CU weight is greater than the maximum inclined weight threshold value. If the decision logic engine determines that the combined CPU and CU weight is greater than the maximum inclined weight threshold value, then control passes to block 1608 . However, if combined CPU and CU weight is not greater than the maximum inclined weight threshold value, then at block 1618 , the decision logic engine determines the “number-of-connected-clients” weight between the BAP and the current AP (this is the fair decision).
- the decision logic engine determines if the weight determined at block 1618 is greater than or equal to the predetermined “accept” threshold value. If the weight determined at block 1618 is greater than or equal to a predetermined “accept” threshold value, then at block 1610 the decision status is set to “accept” (so that the current AP can send a response to the requesting client to allow the requesting client to connect to the current AP. However, if the weight determined at block 1618 is not greater than or equal to the predetermined “accept” threshold value, then at block 1612 the decision status is set to “reject” (so that the current AP can ignore the requesting client probe request).
- the weight determination for each of the parameters is as follows and is based on empirical analysis.
- Weight P +Weight+(( ⁇ x/ 10+((2+( K/ 3)*2)* b/m ))+(((2+( L/ 3)*2)* a/m )+ ⁇ x/ 10))*( ⁇ )
- P can be either negative or positive or zero. P can be used either to increase or decrease the priority of a given parameter. The higher the value of P, the higher the priority. Default value of P is 1. The value P can be changed by the user. The value of P is unique for each parameter, according to certain embodiments.
- m Maximum threshold value of the CU
- ⁇ x ABS (a ⁇ b), which is the absolute value of the difference between current AP channel utilization and Best AP channel utilization.
- Weight P +Weight+(( ⁇ x/ 10+((2+( K/ 3)*2)* b/m ))+(((2+( L/ 3)*2)* a/m )+ ⁇ x/ 10))*( ⁇ )
- the input weight will be always a non-zero value in this equation.
- P can be either negative or positive or zero. P can be used either to increase or decrease the priority of a given parameter. The higher the value of P, the higher the priority. Default value of P is 1. The value P can be changed by the user. The value of P is unique for each parameter, according to certain embodiments.
- ⁇ x ABS (a ⁇ b) (Absolute difference between current AP CPU utilization and Best AP CPU utilization).
- Weight P +Weight+ ⁇ x/S *( ⁇ )+( ⁇ RSSI/30)
- P can be either negative or positive or zero. P can be used either to increase or decrease the priority of a given parameter. The higher the value of P, the higher the priority. Default value of P is 1. The value P can be changed by the user. The value of P is unique for each parameter, according to certain embodiments.
- ⁇ x ABS (a ⁇ b) (Absolute difference between current AP connected clients and Best AP connected clients)
- ⁇ RSSI (a ⁇ b) (A positive difference between current AP RSSI and Best AP RSSI)
- FIG. 17 is a graph that illustrates aspects of the decision logic used for load balancing clients across APs in a wireless environment, according to certain embodiments.
- Graph 1500 also shows the worst case.
- Graph 1700 also shows the Fair 1710 , Inclined 1712 , and Aggressive 1714 decision logic.
Abstract
Third party applications are deployed as “containerized applications” on one or more wireless AP devices. The containerized applications are confined to a pre-allocated segregated disk space within a file system of a wireless AP device. The containerized applications have access to standard Linux services as well as access to advanced features provided by an AP.
Description
- The present invention is continuation-in-part of U.S. patent application Ser. No. 14/938,763, filed Nov. 11, 2015 and entitled “Mobile Application Service Engine,” U.S. patent application Ser. No. 14/749,580, filed Jun. 24, 2015 and entitled WiFi Airtime Allocation,” and U.S. patent application Ser. No. 14/886,636, filed Oct. 19, 2015 and entitled “Distributed Load Balancing for Access Points,” all of which are hereby incorporated by reference herein.
- The present invention is directed to wireless communications, and more specifically to aspects of WiFi network architecture and services.
- The patent or application file contains at least one drawing executed in color. Copies of this patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
-
FIG. 1 shows a high-level diagram that illustrates a cloud-controlled service-ready access point device architecture, according to certain embodiments. -
FIG. 2 illustrates the cloud-controlled service-ready access point device architecture ofFIG. 1 in greater detail, according to certain embodiments. -
FIGS. 3A, 3B and 3C illustrate management of MASE applications, according to certain embodiments. -
FIG. 4 illustrates MASE libraries to support MASE applications installed in a given AP device, according to certain embodiments. -
FIG. 5 illustrates segregation of a portion of the AP file system to confine MASE applications, according to certain embodiments. -
FIG. 6 is a high-level network diagram showing aspects of distributing the consumption of airtime allocated to each connected wireless client device across a cycle time in a wireless communication network, according to certain embodiments. -
FIGS. 7A, 7B, 7C, and 7D are high-level flow charts showing aspects of an airtime calculation engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. -
FIG. 8 is a high-level network flow chart showing aspects of an airtime distribution engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. -
FIG. 9 is a high-level network flow chart showing aspects of a throttler for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. -
FIG. 10 is a high-level network flow chart showing aspects of a feedback engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. -
FIG. 11 is a high-level network diagram showing aspects of a distributed load balancing solution, according to certain embodiments. -
FIG. 12 is a high-level flow chart that illustrates some of the functions of a current AP that receives a probe request, according to certain embodiments. -
FIG. 13 is a high-level flow chart that illustrates some of the operations of the client load balance server, according to certain embodiments. -
FIG. 14 is a high-level flow chart that illustrates some of the features of the ranking operation for ranking the APs, according to certain embodiments. -
FIG. 15 is a graph that illustrates aspects of the ranking logic used for load balancing clients across APs in a wireless environment, according to certain embodiments. -
FIG. 16 is a high-level flow chart that illustrates some aspects of the decision logic engine used by the current AP, according to certain embodiments. -
FIG. 17 is a graph that illustrates aspects of the decision logic used for load balancing clients across APs in a wireless environment, according to certain embodiments. - Methods, systems, user interfaces, and other aspects of the invention are described. Reference will be made to certain embodiments of the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with the embodiments, it will be understood that it is not intended to limit the invention to these particular embodiments alone. On the contrary, the invention is intended to cover alternatives, modifications and equivalents that are within the spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
- Moreover, in the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the invention may be practiced without these particular details. In other instances, methods, procedures, components, and networks that are well known to those of ordinary skill in the art are not described in detail to avoid obscuring aspects of the present invention.
- Mobile Application Service Engine
- According to certain embodiments, a mobile application services engine (MASE) enables the deployment (in clusters or otherwise) of third party applications at access point devices (“AP”). According to certain embodiments, such a deployment of third party applications can be managed remotely using a cloud implemented network management system (cloud-based NMS). Further, the NMS can remotely monitor the activities and status of the third party applications deployed in one or more APs associated with the cloud-based NMS. A wireless access point (AP) is a device that allows wireless devices (client devices) to connect to a wired network using Wi-Fi, or related standards. An enterprise grade AP device includes a chipset that is capable of connecting hundreds of wireless client devices to a wired network at the same time.
- According to certain embodiments, third party applications are deployed on a given AP using software containers (e.g., Linux containers). Such software containers provide a secure environment for the third party applications, while at the same time, protects the AP by confining a given third party application in an isolated file system as described in greater detail herein. The embodiments are not restricted to Linux containers. The type of containers may vary from implementation to implementation.
- According to certain embodiments, such containers preserve the stability and quality of the AP without requiring the customer to override the AP's firmware and/or rebuild the AP's image files. A customer is any establishment or entity that owns or rents APs for deployment in the establishment/buildings. Thus, the customer can be either a retailer or a wireless service provider that deploys the access point device(s).
- According to certain embodiments, applications can be grouped (clustered) such that applications of the same group share the same container in a given AP. Thus, application groups that are installed and run in separate containers can reduce interference and conflicts between the application groups.
- According to certain embodiments, such third party applications in the containers (herein also referred to as “containerized applications” or “MASE applications”) have access to standard Linux services as well as access to advanced features provided by MASE through the MASE application interface (API) on a given AP. For example, standard Linux system services are available to the MASE applications using Linux system calls. Advanced system services are available to the MASE applications through the MASE API.
- According to certain embodiments, a MASE application includes configuration files, any libraries and other binaries needed to run the software application, bundled into the container. According to certain embodiments, a given AP device shares the AP's operating system, CPU and memory with the MASE applications.
- According to certain embodiments, a given AP device includes a MASE run-time management system to manage and monitor the MASE applications. For example, the MASE run-time management system monitors the use of a given AP's resources by each MASE application to prevent hogging of resources by the MASE applications. According to certain embodiments, in order for an AP device to host MASE applications and to host the MASE run-time management system, the AP device includes: 1) WLAN chipset that is capable of connecting at least about 50 wireless client devices at the same time to the wired network that is associated with the AP device (preferably, the WLAN chipset is capable of connecting at least 100 wireless client devices to the wired network that is associated with the AP device), 2) a stand-alone multi-core CPU (central processing unit) that is capable of hosting relevant WiFi related connection services for wireless client devices, as well as capable of hosting/supporting MASE applications, 3) high-speed and large RAM memory capable of hosting MASE applications on the AP device (e.g., RAM size ranges from approximately 1 gigabyte to approximately 4 gigabyte), 4) high-speed and large permanent storage (e.g., permanent storage size ranges from approximately 8 gigabyte to approximately 128 gigabyte) for supporting MASE applications and for storing large amounts of cached data to support better web-services delivery, and 5) dedicated network processor hardware that is capable of analyzing and controlling IP data packets and for controlling data traffic to achieve reasonable wirespeed in the associated wired network but without adversely impacting CPU cycles of the AP's CPU and without adversely impacting the run-time performance of the MASE applications that are running on the AP device.
- According to certain embodiments, third party applications are installed on the one or more APs as MASE application packages using standard ipk packaging (Itsy Package) technology. The embodiments are not limited to ipk packaging technology. The type of packaging technology used may vary from implementation to implementation. An application package includes configuration files, software libraries and executable code, for example.
-
FIG. 1 shows a high-level diagram that illustrates a cloud-controlled service-ready accesspoint device architecture 100, according to certain embodiments.FIG. 1 shows a plurality of cloud-controlled service-ready access point devices (AP) 102 a, . . . ,AP 102 n and a cloud-basedNMS 104. -
AP 102 a . . .AP 102 n are associated with the cloud-basedNMS 104 in a cloud infrastructure that supports the plurality of cloud-controlled service-ready access point devices. Each ofAP 102 a, . . . ,AP 102 n includes arespective container 108 a, . . . , 108 n, which in turn includes arespective application 110 a, . . . , 110 n. Theapplications 110 a, . . . , 110 n are MASE applications because they are containerized at the respective APs. - According to certain embodiments, each AP device can include one container or multiple containers. Further, each container can contain one MASE application or multiple MASE applications. For example, applications can be grouped so that the applications of the same group can share the same container in a given AP. Application groups that are installed and run in separate containers can reduce interference and conflicts between the application groups.
- According to certain embodiments, the cloud-based
NMS 104 has the following functions with respect to the MASE applications: 1) deploy the MASE applications in the one or more APs, 2) manage installation of the one or more MASE applications, 3) upgrade the one or more MASE applications when needed, 4) monitor the running status of the one or more MASE applications, 5) configure the one or more MASE applications, and 6) maintain the one or more MASE applications (e.g., the access to syslog files of MASE applications are implemented through the cloud-based NMS). - According to certain embodiments, the APs use open source Linux (OpenWRT). According to certain embodiments, third party developers use the MASE API and the MASE software development kit (SDK, which is based on the OpenWRT build environment) to develop the MASE application. The MASE SDK includes tools for third party developers to build MASE application packages and to download the application packages to the APs. The embodiments are not limited to the Linux OpenWRT operating system. The type of operating system used for the AP may vary from implementation to implementation. The MASE applications can be uploaded to the cloud-based NMS. According to certain embodiments, the Cloud based NMS provides an interactive GUI (graphical user interface) to facilitate the upload process.
- According to certain embodiments, cloud-based
NMS 104 is the network management system that manages clusters of APs. Cloud-basedNMS 104 is responsible for the full cycle deployment of third party applications such that the applications can be managed remotely via cloud-basedNMS 104 without any need of performing administrative tasks locally on the APs. - According to certain embodiments, MASE applications can be effectively “walled-off” by confining them to a pre-allocated a sub-file system of limited size within the AP file system as explained in greater detail herein with reference to
FIG. 5 , herein. -
FIG. 2 illustrates the cloud-controlled service-ready accesspoint device architecture 200 in greater detail, according to certain embodiments. For purposes of convenience,FIG. 2 shows only one of the APs of the plurality of APs. - In
FIG. 2 , cloud-basedNMS 204 includes a cloud-basedcontroller 250 and MASE applications that can be downloaded and installed onAP device 202. As non-limiting examples, the MASE applications stored inNMS 204 includemedia applications 252,analytics applications 254, customizedapplications 256, as well as various applications A-N inapplication marketplace 258 from which a customer can choose to download one or more applications A-N onto theAP device 202. - For purposes of explanation, assume that a customer that owns or leases
AP device 202 would like to have several MASE applications downloaded toAP device 202. - According to certain embodiments, AP device (AP) 202 communicates with cloud-based
NMS 204 through a secure tunnel inInternet 206, for example. - According to certain embodiments, non-limiting examples of characteristics of
AP device 202 that include operating system services 220 (e.g., Linux system services),web service 222, deep packet inspection (DPI)service 224,application configuration 226, NMS events interface 228, operating system calls 230, MASE application programing interface 232 (MASE API), MASE application 210 (downloaded from cloud-based NMS 204). According to certain embodiments, an application configuration that is specific to an application for a specific AP or cluster of APs can be specified at the NMS and then downloaded to the specific AP or cluster of APs. The MASE API on the AP delivers the configuration to the relevant application using callbacks that the given MASE application registers with the MASE API, for example. - According to certain embodiments,
AP device 202 also includes physical resources on the AP device such as a network processor, a WLAN chipset capable of connecting more than 50 wireless clients to a wired network at the same time, an AP RAM memory, permanent storage, AP multi-core central processing units (CPU), and an AP file system. Such physical resources are on the AP device. -
Application marketplace 258 is a repository of third party applications that are developed using the MASE technology, according to certain embodiments. - Third party application developers use the MASE software development kit (SDK) to develop MASE applications that can be deployed in a given AP device. When the third party application developers upload their MASE applications to
application marketplace 258, the third party application developers can define the type of MASE APs where the applications can be deployed. The third party application developers can define their applications as private or public applications. Private applications can only be deployed at APs that are managed by the same AP providers that the third party developers are associated with. Public applications can be deployed at any of the APs, and a licensing scheme is available for the third party developer to collect licensing fees from AP owners that deploy the public applications, according to certain embodiments. - Further, third party application developers can define the type of configuration that can be applied to the applications, such as the name of the configuration parameters, the type of the parameters and the range or possible values of the parameters. A configuration can be defined for a group of APs where the application is deployed, or a configuration can be defined for a specific AP.
- According to certain embodiments, the MASE SDK includes the open source OpenWRT build environment, with which the third party developer can use to build the application and package the application in the standard ipk package. The SDK also includes the MASE API library and the header file, which can be included in the application if the third party developer decides to leverage the system services provided by the MASE API.
- Further, the SDK includes a suite of tools that allow the third party developer to manage the installation of the applications of the third party developer on a MASE AP device as part of the development test bed. The MASE API is required to run in a developer mode, according to certain embodiments. When the developer mode is enabled, the AP handles the installation commands generated by the tool suite.
- According to certain embodiments, AP owners can select MASE applications to be deployed on their APs. For example, AP owners can select either private applications that are developed by their own developers, or public applications that are developed by any third party developers. When the applications are selected to be deployed or installed on some APs, configurations can be specified for the group of APs based on the requirements of the particular deployments. Further, the AP owners can decide whether the applications, after they are installed on the APs, should start or stop running, or whether they should be restarted. The AP owners can do so by specifying the running status of the applications.
- According to certain embodiments, the cloud-based
NMS 204 implements different interfaces and tools for application maintenance. For example, the status and health of the MASE applications (as well as memory usage, CPU usage and disk usage) can be monitored at cloud-basedNMS 204. The application status and health are updated consistently. - As another example, if
MASE application 210 has crashed and is restarted, events are generated and can be reviewed at the NMS event log at cloud-basedNMS 204. Application syslog message files are uploaded to the cloud-basedNMS 204 and can be retrieved for review, for example. - According to certain embodiments,
MASE applications 210 running in containers on the AP have access to standardLinux system services 220 such as the file system and internetworking features (for example, TCPIP protocol, UDP protocol, HTTP server connections, and various software libraries). In addition, advanced features such as Web services are available to the MASE applications via MASE API. Further, theMASE API 232 supports cloud-based application configuration and event logging. The embodiments are not limited to Linux operating systems. The type of operating systems may vary from implementation to implementation. - Third
party MASE applications 210 deployed at APs have access to all standard Linux system services 220.Linux system services 220 include but are not limited to: 1) Pre-allocated disk space for a variety purposes such as application data storage, 2) IP internetworking through UNIX socket API, and 3) Standard UNIX syslog service. - According to certain embodiments,
MASE applications 210 are installed and run within the confinement ofcontainer 208.Container 208 is constructed based on Linux technologies, according to certain embodiments. For example,container 208 uses the same standard Linux tools and libraries built in the AP image from the OpenWRT distribution. EachMASE application 210 has pre-allocated disk space in an isolated file system inside the container. EachMASE application 210 has full read and write access to its dedicated pre-allocated disk space. However, theMASE application 210 has only read access outside its dedicated pre-allocated disk space within the container when running as a non-privileged user. The construction ofcontainer 208 may be based on other technologies and is not limited to Linux technologies and may vary from implementation to implementation. - According to certain embodiments,
MASE applications 210 are packaged using the standard ipk package format that is used by the open source OpenWRT build environment. Application installation at a given AP is managed through the cloud-basedNMS 204 remotely. Standard application installation scripts (e.g., preinst, postinst, prerm, postrm) are supported and invoked as applications are installed or uninstalled at a given AP, according to certain embodiments. - According to certain embodiments, the running status of a
MASE application 210 is managed remotely at cloud-basedNMS 204. TheMASE application 210 is started or stopped using a service status management mechanism via application “init” script that is installed at location /etc/init.d/, for example. Init script commands, e.g., start, stop, restart, status, are used to manage the application status. - According to certain embodiments, while
MASE applications 210 are running within the confinement of acontainer 208 on the AP, theMASE applications 210 have full access to other system resources on the AP such as the physical memory and CPU cycles of the AP. According to certain embodiments, a givenMASE application 210 can specify the amount of system resources that the given MASE application needs in the application package manifest. According to certain embodiments, the amount of system resources are capped at a hard limit when theMASE application 210 is installed on the one or more APs.MASE applications 210 are more or less continuously monitored and can be restarted as soon as they are found to have used up too many AP resources based on the predefined limits, according to certain embodiments. - According to certain embodiments, application configuration is managed at cloud-based
NMS 204. However, theconfiguration information 226 is eventually passed to theMASE application 210 via theMASE API 232. If theMASE application 210 requires configuration support from cloud-basedNMS 204, theMASE application 210 can use theMASE API 232 in order to receive the application configuration information. - Web applications are server side applications that implement specific Web requests from wireless clients. According to certain embodiments, there are three models of web applications for accessing the web requests. According to certain embodiments, the three models of web applications for accessing the web requests are: 1) Proxy service model based on domain names specified by applications, 2) Static web content model, 3) FCGI applications model.
MASE API 232 provides advanced HTTP proxy service based on proxy requirements specified by theMASE application 210. When HTTP proxy service is enabled for certain domain names, the HTTP requests are proxied and forwarded to proxy HTTP/TCP ports that theMASE application 210 specifies. TheMASE application 210 can implement an HTTP server on such proxy HTTP/TCP ports to service the proxied HTTP requests received at those HTTP/TCP ports. - According to certain embodiments,
Web service 222 can host static Web content packaged in the application ipk package. Third party developers can package Web content, such as HTML pages, JavaScript programs, and multimedia files (pictures/audio/video) in the MASE application package together with a Web hosting manifest that specifies the base URL locations (including host names and base paths) for the Web content. When theMASE application 210 is installed,Web service 222 is reconfigured to support the URL locations specified in the Web hosting manifest. - According to certain embodiments, third party applications can implement the standard fast common gateway interface (FCGI). The FCGI applications are installed in the same Linux container environment as regular third party MASE applications on the APs, but are invoked as FCGI applications with
Web service 222 handling the Web request before invoking the third party applications via the standard FCGI protocol.Web service 222 can be configured to invoke the third party applications for selected URLs based on FCGI Web hosting manifest, which is packaged together with the third party FCGI application in the application ipk package. For example, when a client device requests a web page or a video from the internet,web service 222 will check to see if there is a match in the URL list. If there is a match,web service 222 invokes the FCGI application, which in turn, will analyze the request from the client device and decide on the appropriate response to send to the client device. - According to certain embodiments,
DPI service 224 supports DPI related applications by providing advanced flow-based packet sniffing. Packets can be captured on a per flow basis, with adjustments allowed to sniff more or fewer packets for a particular flow. In addition, DPI analytic applications can use theMASE API 232 to export statistics and reports to the cloud-basedNMS 204. Cloud-basedNMS 204 can provide cloud-based storage and GUI integration for display purposes. - According to certain embodiments,
MASE API 232 allows the third party application to access various system services. For example, application configuration is managed at cloud-basedNMS 204, which sends the configuration information to the relevant AP where the application is deployed. The configuration information is sent to the application viaMASE API 232. As another example, for all the HTTP requests that theMASE application 210 would likeWeb service 222 to proxy on behalf of theMASE application 210, theMASE application 210 can specify the domain names of such URLs via theMASE API 232 together with the proxy HTTP/TCP port number. TheMASE API 232 reconfigures the networking stack on the AP to proxy the HTTP requests destined for such domain names, and forwards the requests to the specified proxy HTTP/TCP ports. The third party application can implement a Web server on such proxy HTTP/TCP ports to receive the proxied HTTP requests. Further, flow-based packet sniffing is supported to capture packets for DPI analysis. Statistics and reports are exported to cloud-basedNMS 204 for integration with cloud-based services provided by cloud-basedNMS 204. In addition, cloud-basedNMS 204 implements an event log to log various kinds of system events related to activities at the APs.MASE applications 210 can generate events usingMASE API 232 so that such events can be logged at the event log in cloud-basedNMS 204. -
FIGS. 3A, 3B and 3C illustrate management of MASE applications, according to certain embodiments.FIG. 3A showsMASE application environment 300A on an AP.FIG. 3A showscontainer 302 that includes anapplication manager agent 304, andMASE application 308.Application manager agent 304 communicates with the AP'sapplication manager daemon 312 via inter-process communication (IPC) 316. For example,application manager agent 304 monitors CPU usage, memory usage (e.g., RAM usage), and disk usage of theMASE application 308.Application manager daemon 312 handles problems locally at the AP based on pre-determined policies, for example.Application manager agent 304 communicates information 322 (such as application running status, health status) toapplication manager daemon 312.Application manager agent 304 also communicates relevant information withMASE application 308.MASE application 308 usesMASE API 310 to communicate information 320 (e.g., application events) toapplication manager daemon 312 viaMASE manager API 314.MASE application 308 receives application configuration information 318 fromapplication manager daemon 312 viaMASE manager API 314 andMASE API 310. Further,application manager daemon 312 communicatesevent information 326 to themonitor daemon 328 on the AP. Also,application manager daemon 312 can receive information 324 (e.g., installation configuration information, application status configuration information, application configuration information) from the cloud-based NMS, according to certain embodiments. -
FIG. 3B shows a different embodiment ofapplication manager agent 304 b.Application manager agent 304 b includes aMASE DPI API 307 that can receive information such as application configuration and application status configuration (318, 318 b) from MASEmanager DPI API 314 b. Further,application manager agent 304 b usesAPI 309 to communicate (e.g., DPI information) withMASE application 308.MASE application 308 usesMASE DPI API 310 b to send buffer file upload 320 b toapplication manager daemon 312 through MASEmanager DPI API 314 b. -
FIG. 3C shows a different embodiment ofMASE application 308 c, which usesMASE API 310 to communicate transparentproxy domain information 320 c toapplication manager daemon 312 viaMASE manager API 314. -
FIG. 4 illustrates MASE libraries to support MASE applications installed in a given AP device, according to certain embodiments.FIG. 4 shows application manager library layers 402 and Application/application manager agent library layers 404 in aMASE library architecture 400, according to certain embodiments. Application manager library layers 402 are not exposed to third party developers and comprise a privatedpi library layer 406, and a privateAPI library layer 412. Application/application manager agent library layers 404 comprise a receive-only DPI library layer 408 a send-onlyDPI library layer 410, and anAPI library layer 414, according to certain embodiments.FIG. 4 also shows acommunication transport layer 416 for IPC messages, according to certain embodiments. -
FIG. 5 illustrates segregation of a portion of the AP file system to confine MASE applications, according to certain embodiments.FIG. 5 shows thefile system 500 of a given AP. According to certain embodiments,file system 500 comprises: 1) selecteddirectories 502 that can be shared with the MASE application container, viahard links 520, 2)proprietary directories 504 that are not shared with the MASE application container, and 3) thecontainer file system 506.Container file system 506 uses change root (chroot) for all guest processes (directories 508, and 510). According to certain embodiments, even though theproprietary directories 504 are not shared with the MASE application container, theproprietary directories 504 include a virtual disk partition in the form of a disk image that contains the writeable disk space to which a given MASE application container can write data.Mount 522 is a Linux process for making the virtual disk partition accessible by the MASE applications through regular Linux filesystem management commands (e.g., commands related to creating files, writing to a file, creating subdirectories, etc.). In other words, the MASE applications on the AP device has access to such a virtual disk partition. The embodiments are not limited to Linux-type file systems. The file systems may vary from implementation to implementation. - According to certain embodiments, a method of WiFi networking comprises: deploying containerized applications (MASE applications) on a wireless access point device of a plurality of wireless access point devices that are associated with a WiFi network for connecting a plurality of wireless client devices to a wired network. The method comprises using at least one application container on the AP device. The method further comprises installing one or more third party applications in the at least one application container on the AP device. The method further comprising using an AP device that includes: 1) a WLAN chipset on the AP device, which WLAN chipset is capable of connecting at least about 50 wireless client devices at the same time to the wired network that is associated with the AP device (preferably, the WLAN chipset is capable of connecting at least 100 wireless client devices to the wired network that is associated with the AP device), 2) a stand-alone multi-core CPU (central processing unit) on the AP device, which stand-alone multi-core CPU is capable of hosting relevant WiFi related connection services for wireless client devices, as well as capable of hosting/supporting MASE applications, 3) a high-speed and large RAM memory on the AP device, which RAM memory is capable of hosting MASE applications on the AP device, 4) high-speed and large permanent storage on the AP device for supporting MASE applications and for storing large amounts of cached data to support better web-services delivery, 5) dedicated network processor on the AP device, which dedicated network processor is capable of analyzing and controlling IP data packets and for controlling data traffic to achieve reasonable wirespeed in the associated wired network but without adversely impacting CPU cycles of the AP's CPU and without adversely impacting the run-time performance of the MASE applications that are running on the AP device. The method further comprises using a pre-allocated segregated disk space within a file system on the AP device that can be accessed and used by the containerized applications that are installed on the AP device. The method further comprises using an application run-time management system on the AP device to monitor the execution of the containerized third party applications and including monitoring the CPU usage, RAM usage, disk storage usage, application running status, application health status, and application events. The method further comprises using a MASE application programming interface (MASE API) on the AP device to provide web services to the MASE applications installed on the AP device and to provide deep packet inspection services to the MASE applications installed on the AP device.
- WiFi Airtime Allocation
- According to certain embodiments, the consumption of airtime allocated to each connected wireless client device (also referred to herein as a connected wireless node) of at least a subset of wireless client devices/nodes that are connected to corresponding access point devices in a wireless system is distributed across a cycle time. According to certain embodiments, a cycle time is a predetermined value and is the duration of time per iteration.
- According to certain embodiments, the consumption of airtime that is allocated to each wireless client device/node is distributed across a cycle period based on calculating the airtime required for the given wireless client device/node of a plurality of wireless client devices/nodes for transmitting a packet when connected to a corresponding access point device in the wireless communication system.
- According to certain embodiments, the airtime allocated to each connected wireless client device/node is estimated based on comparing the required airtime for the respective connected wireless client device/node with the available airtime for each connected wireless client device/node, where available time is the total available airtime in the cycle divided by the total number of connected wireless nodes.
- According to certain embodiments, the consumption of airtime that is allocated to each wireless client device is distributed across a cycle period based on comparing the consumed airtime of the given wireless client device/node with the elapsed cycle time when the given wireless client is connected to a corresponding access point device in the wireless communication system.
- According to certain embodiments, the distribution of consumption of airtime allocated to each connected wireless client device/node across a cycle period in the wireless communication system of a plurality of wireless client devices/nodes includes throttling transmission of packets of the given connected wireless client device/node based on one or more pre-determined criteria.
- According to certain embodiments, the distribution of consumption of airtime allocated to each connected wireless client device/node across a cycle period in the wireless communication system of a plurality of wireless client devices/nodes includes providing feedback information including computation of airtime transmission of a given packet by the given connected wireless client device/node through its associated access point device.
- A given connected wireless client device/node in a wireless communication system can consume its corresponding allocated airtime per cycle all at once in the cycle. Such a connected wireless client device/node can result in hogging the transmission channel at its associated access point device. Thus, according to certain embodiments, a given connected wireless client device/node in a wireless communication system is controlled to consume its allocated airtime in a controlled distributed manner across a cycle to prevent excessive hogging of a transmission channel and/or to prevent excessive dropping of packets.
- According to certain embodiments, such a wireless communication system periodically estimates the airtime required by a given connected wireless node and throttles the given node's transmission of packets across a given cycle based on the usage of airtime per cycle. Further, such a wireless communication system provides correction feedback information per cycle to the airtime estimation engine.
-
FIG. 6 is a high-level network diagram showing aspects of distributing the consumption of airtime allocated to each connected wireless client device across a cycle time in a wireless communication network, according to certain embodiments. InFIG. 6 , atblock 602, the airtime estimation engine calculates the node allocated airtime for a given connected wireless node of at least a subset of a plurality of wireless nodes connected to a corresponding access point in the wireless communication network to produce the corresponding node allocated airtime result atblock 604. Node allocated airtime is the duration of time allocated to a connected wireless node for receiving and transmitting packets per cycle time, according to certain embodiments. Atblock 608, an airtime distribution engine receives an elapsed cycle time percentage value as an input fromblock 606 and also receives the node allocated airtime result fromblock 604 and uses such inputs for determining whether to send a packet that the given connected wireless node is attempting to send (or receive) to a throttler atblock 610 or to send the packet to the wireless transceiver atblock 612.FIG. 6 also shows some functional aspect of the wireless transceiver at the physical layer shown atblock 614. At the physical layer, the wireless transceiver adds the time that the given connected wireless node used to transmit (616) a given packet with the time the given connected wireless node used to receive (618) a packet and send the information to afeedback engine 620. Thefeedback engine 620 updates a feedback table 622. For example, the feedback table includes information such as number of transmission retries, data rate, unused airtime, drop airtime, transmission airtime, receiving airtime, Request-to-Send (RTS) flags, Clear-to-Send (CTS) flags etc, according to certain embodiments. Atblock 624, the feedback engine send information on the airtime consumed by the given connected wireless node to the airtime estimation engine. The process described with reference toFIG. 6 is repeated for each connected wireless node of at least a subset of a plurality of wireless nodes connected to the corresponding access point in the wireless communication network, according to certain embodiments. -
FIGS. 7A, 7B, 7C, and 7D are high-level flow charts showing aspects of an airtime calculation engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. The processes inFIGS. 7A, 7B, 7C, and 7D are described with respect to a given connected wireless node and are repeated for each connected wireless node of at least a subset of the plurality of wireless nodes connected to the corresponding access point in the wireless communication network, according to certain embodiments. - At
block 702 inFIG. 7A , the airtime estimation engine calculates the total available airtime per cycle for a given connected wireless node. According to certain embodiments, the total available airtime=(100−channel congestion)×cycle time, according to certain embodiments. For example, the channel congestion is the amount of channel capacity that is currently in use and therefore not available for transceiving packets for the given connected wireless node. Further, atblock 702, total free airtime=total available airtime. Atblock 704, the airtime estimation engine determines the total number of wireless nodes connected to the given access point (referred herein as “connected wireless nodes”) in the wireless communication network. Atblock 706, the airtime estimation engine calculates the airtime than can be shared by the connected wireless nodes. Such an estimate is referred to as a node initial airtime. Node initial airtime=total available airtime/number of connected wireless nodes, according to certain embodiments. Atblock 708, for the given connected wireless node, the airtime estimation engine calculates the airtime used by the given connected wireless node (also referred to as “node used airtime” or “node consumed airtime”) in the previous cycle. The node consumed airtime is provided by the feedback engine as explained in greater detail herein. The node used airtime will be zero for wireless nodes that are newly connected to the access point in the current cycle. According to certain embodiments, atblock 708, the airtime estimation engine makes the following calculation: node used airtime=airtime consumed by the given connected node for transmitting packets+airtime consumed by the given connected node for receiving packets+airtime consumed by the given connected node due to re-transmissions+estimated airtime consumed by the given connected node for packets dropped. - Re-transmissions are also known as retries. A WiFi medium is not a deterministic medium because of channel interference. There may be many retries in the channel due to interference. Retries are the number of times a connected wireless node tries to transmit a data packet until the packet is successfully transmitted. For example, a successful transmission is when the connected wireless node receives an acknowledgment for each of the data packets it has transmitted. A given connected wireless node will retry sending packets that are not acknowledged by the receiver. Retries are more frequent when the WiFi channel is congested, or if the given connected wireless node is far away from the access point and/or is not in the line of sight of the access point. In some wireless chipsets of wireless nodes, the number of retries may be pre-programmed for every packet so that the packets are resent automatically.
- A connected wireless node that attempts many retries will occupy much more airtime than a connected wireless node that is able to send data packets successfully with few or no retires. RTS/CTS (Request to Send/Clear to Send) comprise an optional mechanism used by the 802.11 wireless networking protocol to reduce frame collisions introduced by wireless node problems. As the number of retries grows, the rate controller of the WiFi system will assume that the link quality is bad and will try to protect each frame with RTS/CTS messages. The amount of airtime consumed by RTS/CTS messages needs to be accounted for with respect to the given connected wireless node that is attempting to send the data packet. For purposes of illustration, assume that connected Wireless Node1 and Wireless Node2 are attempting to send packets. Further assume that Wireless Node1 is sending packets with RTS/CTS exchange mechanism. The airtime consumed by Wireless Node1 will be higher than that of Wireless Node2.
- At
block 710 ofFIG. 7A , the airtime estimation engine calculates the error correction based on the inputs for the previous cycle. The error correction is needed to calculate the airtime required by the given connected wireless node for transmitting/receiving packets. According to certain embodiments, error correction=node used airtime−previous cycle node allocated airtime. - At
block 712 ofFIG. 7A , the airtime estimation engine calculates the airtime required by the given node in the current cycle to transmit/receive packets. According to certain embodiments, the node required airtime=previous cycle node allocated airtime+error correction. Control of the process passes to block A1 ofFIG. 7B . - At
block 720 ofFIG. 7B , the airtime estimation engine determines, for the given connected wireless node, if the node required airtime is greater than the node initial airtime. If the node required airtime is greater than the node initial airtime, then atblock 724, the given node is allocated the node initial airtime. Atblock 730, the total free airtime is calculated again as: total free airtime=total free airtime−node allocated airtime. Atblock 734, the airtime estimation engine determines if there are more connected wireless nodes for which airtime needs to be estimated. If there are more connected wireless nodes for which airtime needs to be estimated, then control passes to block A2 ofFIG. 7A . From block A2 ofFIG. 7A , control passes back block 704 that is previously described herein with reference toFIG. 7A . If there are no more connected wireless nodes for which airtime needs to be estimated, then control passes to block B ofFIG. 7C as described herein. - If at
block 720, it is determined that the node required airtime is not greater than the node initial airtime, then atblock 722, the node allocated airtime is set as equal to the node required airtime, according to certain embodiments. Next, atblock 726, it is determined if the node allocated airtime is less than the node minimum airtime threshold. The node minimum airtime threshold is a predetermined value, according to certain embodiments. If it is determined that the node allocated airtime is not less than the node minimum airtime threshold, then control passes to block 730.Block 730 is previously described above. If it is determined that the node allocated airtime is less than the node minimum airtime threshold, then atblock 728, the node allocated airtime is set as equal to node minimum airtime threshold, in order to avoid a zero value, according to certain embodiments. - As shown in
FIG. 7C , control passes from block B to block 740 where it is determined if the remaining airtime is greater than zero. If it is determined that the remaining airtime is not greater than zero then the process endsblock 742. If it is determined that the remaining airtime is greater than zero, then each wireless node connected to the access point is considered atblock 744. Atblock 746, for the connected wireless node under consideration atblock 744, it is determined if the node required airtime is greater than the node allocated airtime. If atblock 746, if it is determined that the node required airtime is greater than the node allocated airtime, then atblock 748, the node share for the given node is calculated as: node share=remaining airtime×(estimated airtime of packets dropped by the given node/total estimated airtime of packets dropped by the given access point). Atblock 750, the airtime estimation engine calculates the node allocated airtime as: node allocated airtime=node allocated airtime+node share, according to certain embodiments. According to certain embodiments, the purpose of a “node share” is to share the unused airtime among connected wireless nodes that need more airtime. Next, control passes to block B1 ofFIG. 7D . If atblock 746, it is determined that the node required airtime is not greater than the node allocated airtime, then control passes back to block 744 where the airtime estimation moves to the next connected wireless node for consideration as described with reference to block 746. - As shown at block B1 of
FIG. 7D , control passed to block 760 where the airtime estimation engine calculates the remaining airtime for the given connected wireless node is calculated as: remaining airtime=remaining airtime−node share. Atblock 762, it is determined if there are more connected wireless nodes to be considered. If there are more connected wireless nodes to be considered, then control passes back to block B2 ofFIG. 7C . As shown inFIG. 7C , from block B2, control passes to block 744 that is previously described herein with reference toFIG. 7C . - If at
block 762, it is determined that there are no more connected wireless nodes to be considered, then atblock 764, it is determined if the remaining airtime is greater than zero. If it is determined that the remaining airtime is not greater than zero, then the process ends atblock 766. If however it is determined atblock 764 that the remaining airtime is greater than zero, then atblock 768 each connected wireless node is considered for calculating its node share atblock 770 as: node share=remaining airtime/number of wireless nodes connected to the access point. Atblock 772, the airtime estimation engine sets the node allocated airtime for the given connected wireless node under consideration as: node allocated airtime=node allocated airtime+node share. Further atblock 772, the airtime estimation engine sets the remaining airtime as: remaining airtime=remaining airtime−node share. Atblock 772, it is determined if there are more connected wireless nodes to be considered. If there are no more connected wireless nodes to be considered, then the process ends atblock 776. If however, there are more connected wireless nodes to be considered, then control passes back to block 768. -
FIG. 8 is a high-level network flow chart showing aspects of an airtime distribution engine for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. When a data packet associated with a given connected wireless node is ready to be transmitted atblock 802, the airtime distribution engine finds the destination wireless node atblock 806. Atblock 808, the airtime distribution engine calculates the consumed airtime for the given destination wireless node found atblock 806 as: destination node consumed airtime=airtime consumed by the destination node for transmitting packets+airtime consumed by the destination node for receiving packets+airtime consumed by the destination node due to re-transmissions. The airtime consumed by the destination node for transmitting packets, the airtime consumed by the destination node for receiving packets, and the airtime consumed by the destination node due to re-transmissions information can be obtained from the feedback table. Atblock 810, the airtime distribution engine calculates the destination node consumed airtime percentage as: destination node consumed airtime percentage=(destination node consumed airtime/destination node allocated airtime)×100. Atblock 812, the airtime distribution engine calculates the elapsed cycle time percentage as: elapsed cycle time percentage=(total airtime consumed by all the connected wireless nodes in the cycle/total available airtime)×100. Atblock 814, the airtime distribution engine determines if the elapsed cycle time percentage minus the destination node consumed airtime percentage is greater than the throttle threshold value. The throttle threshold value is a predetermined value according to certain embodiments. If it is determined that the elapsed cycle time percentage minus the destination node consumed airtime percentage is not greater than the throttle threshold value, then atblock 818, the given data packet is queued to the throttler. However, if it is determined that the elapsed cycle time percentage minus the destination node consumed airtime percentage is greater than the throttle threshold value, then atblock 816, the airtime distribution engine calculates the required airtime for transmitting the given data packet (“required packet airtime”) based on information for the feedback table. According to certain embodiments, the required packet airtime is a function of packet length, selected data rate, previous retries for the selected data packet rate obtained from the feedback table. Atblock 820, the airtime distribution engine determines if the required airtime for transmitting the given data packet is greater than the destination node's remaining airtime. If it is determined that the required airtime for transmitting the given data packet is greater than the destination node's remaining airtime, then control passes to block 818 where the given data packet is queued to the throttler. However, if it is determined that the required airtime for transmitting the given data packet is not greater than the destination node's remaining airtime, then the data packet is sent to the wireless transceiver for transmission. The airtime distribution process ends atblock 824. -
FIG. 9 is a high-level network flow chart showing aspects of a throttler for wireless nodes client devices connected to wireless access point devices in a wireless network, according to certain embodiments. InFIG. 9 , block 902 operates on the next packet (current packet) in the throttler queue. Atblock 904, the throttler determines if the continuous dropped packet count of a given node associated with the current packet is greater than the dropped packet count threshold. According to certain embodiments, the dropped packet count threshold is a predetermined value. If it is determined that the continuous dropped packet count of the given node associated with the current packet is greater than the dropped packet count threshold, then the packet is sent to the wireless transceiver atblock 906. However, if it is determined that the continuous dropped packet count of a given node associated with the current packet is not greater than the dropped packet count threshold, then atblock 910, it is determined if the given current packet is an aggregate packet. If it is determined that the given current packet is an aggregate packet, then atblock 918, the size of the aggregate packet is limited by consulting the feedback table atblock 920. Atblock 922, it is determined if the size of the aggregate packet is greater than the minimum required aggregate size. If it is determined that the size of the aggregate packet is not greater than the minimum required aggregate size, then atblock 924, the packet is dropped. - However, if it is determined that the size of the aggregate packet is greater than the minimum required aggregate size, then at
block 926, it is determined if the transmission time for the aggregate packet is less than the remaining airtime of the destination node. If it is determined that the transmission time for the aggregate packet is less than the remaining airtime of the destination node then atblock 930, the aggregate packet is sent to the wireless transceiver. However, if it is determined atblock 926 that the transmission time for the aggregate packet is not less than the remaining airtime of the destination node, then atblock 928, the aggregate packet is dropped. - If at
block 910, it is determined that the current packet is not an aggregate packet, then atblock 912, it is determined if the packet is a UDP packet. If it is determined that the packet is a UDP packet, then atblock 916, the UDP packet is queued for transmission during the idle cycle time if there is an un-utilized airtime available at the end of cycle. According to certain embodiments, a decision may be made to drop the UDP packet if the number of packets queued thus far for the given connected wireless node exceeds a predetermined maximum value. When queuing each of the packets, the throttler will track the time required to transmit each of the packets based on the size of the packet, according to certain embodiments. As the time elapses in the current cycle, if the connected wireless nodes become idle (for example, there are no packets to be transmitted for any of the connected wireless nodes), then the throttler will transmit the queued UDP packets until one of the connected wireless nodes makes a transmission request. At the end of the cycle period, the packets remaining in the UDP queue will be dropped, according to certain embodiments. - However, if a
block 912, it is determined that the packet is not a UDP packet (e.g., it is a TCP packet), then atblock 914, the packet is dropped. The process ends atblock 908. - At the end of each cycle time, the feedback engine provides feedback to the airtime estimation engine. The feedback information can include the number of packet dropped, the time take for transmission, the time wasted in retries and the time consumed for receiving the packets. According to certain embodiments, the feedback engine can be queried by the airtime distribution engine for the required airtime to transmit a packet.
- According to certain embodiments, the airtime of a packet is computed at the completion of transmission of each packet. At the completion of transmission, the following information can be obtained from the access point hardware for computing airtime:
-
- 1. Size of the packet
- 2. Data Rate1, Number of tries1, RTS/CTS Flags
- 3. Data Rate2, Number of tries2, RTS/CTS Flags
- 4. Data Rate3, Number of tries3, RTS/CTS Flags
- 5. Data Rate4, Number of tries4, RTS/CTS Flags
- According to certain embodiments, the access point hardware decides the rate of transmission for packets associated with each wireless node. According to certain embodiments, the feedback engine builds a lookup table referred to as a feedback table. As a non-limiting example, the feedback table includes supported PHY layer statistics such as:
-
RATE1, RATE2, . . . , RATEN RATE1 TX PKTS, RATE2 TX PKTS, . . . RATEN TXPKTS RATE 1TX RETRIES, . . . , RATEN TX RETRIES Unused Airtime Transmission Airtime Receiving Airtime Drop Airtime TX = transmission, PKTS = packets. -
FIG. 10 is a high-level network flow chart showing aspects of a feedback engine for wireless client devices/nodes connected to wireless access point devices in a wireless network, according to certain embodiments. Atblock 1002 inFIG. 10 , for each transmitted packet, the feedback engine obtains the rate of transmission information of the transmitted packet from the access point hardware. Atblock 1004, the feedback engine obtains the number of retries associated with the transmission rate information obtained atblock 1002. Atblock 1006, the feedback engine obtains the packet transmit airtime from the access point hardware. Atblock 1008, the node transmitted airtime is updated by the feedback engine. Atblock 1010, the feedback table is updated by the feedback engine. The process ends atblock 1012. - According to certain embodiments, a wireless communication system comprises an airtime estimation engine for determining a respective node allocated airtime in a cycle time period for a given wireless node connected to an access point in the wireless communication system and is based on comparing a required airtime for the given wireless node with an available airtime for the given wireless node in the wireless communication system. According to certain embodiments, the available time is a total available airtime in a cycle divided by a total number of connected wireless nodes in the wireless communication system. According to certain embodiments, such a system further comprises an airtime distribution engine for controlling the given wireless node's consumption of its node allocated airtime across the cycle time period and is based on comparing a consumed airtime percentage of the given wireless node with an elapsed cycle time percentage of the cycle time period.
- According to certain embodiments, the wireless communication system further comprises a throttler engine for determining a packet type of each respective packet in a queue of the throttler engine. According to certain embodiments, the throttler engine queues the respective packet for transmission during idle cycle time if the packet type is determined to be a UDP type.
- According to certain embodiments, the throttler engine limits packet size if the packet type of the respective packet is determined to be an aggregate type. According to certain embodiments, the throttler engine drops the respective packet if the packet type is determined to be a TCP type.
- According to certain embodiments, the airtime distribution engine calculates a required packet airtime for a respective packet based on information from a feedback table. According to certain embodiments, the feedback engine obtains a respective packet transmission time.
- According to certain embodiments, the wireless communication system further comprises a feedback engine for obtaining a respective packet transmission rate. According to certain embodiments, the feedback engine obtains a respective packet transmission time. According to certain embodiments, the feedback engine updates a feedback table.
- Distributed Load Balancing for Access Points
- According to certain embodiments, in high density wireless environments where there may be a large number of wireless client devices that are competing for access points, load balancing across access points in the wireless environment may be achieved using a cloud based, controller-less, distributed solution, according to certain embodiments.
- According to certain embodiments, a distributed solution of load balancing across access points has a faster convergence rate than non-distributed solutions.
- According to certain embodiments, a distributed solution of load balancing across access points obviates a single point of failure.
- According to certain embodiments, a distributed solution of load balancing across access points reacts more efficiently to changes in the wireless environment.
- According to certain embodiments, a distributed solution of load balancing across access points in a cloud based, controller-less wireless environment helps reduce capital expenditure, operational expenditure and reduces complexity of the wireless environment.
- According to certain embodiments, a distributed solution of load balancing includes using at least of subset of the following:
-
- a) Visibility of a wireless client device (also referred to as a “client”) to one or more access points in the wireless environment;
- b) Distance of the client from an access point;
- c) Congestion in the channel associated with the access point;
- d) CPU load of an access point;
- e) Number of existing clients connected to an access point;
- f) Received Signal Strength Indication (“RSSI”) of the client as seen by an access point.
- According to certain embodiments, when a client would like to connect to an access point, the client (also referred to as a “requesting client”) sends, to access points that are nearby, a probe request to connect. According to certain embodiments, each access point that receives a probe request from the requesting client obtains information on the best access point for connecting to the requesting client.
- According to certain embodiments, an access point decides to respond to the requesting client based on at least the information on the best access point for connecting to the requesting client. According to certain embodiments, an access point decides to respond to the requesting client based on one or more criteria from a set of predetermined criteria. According to certain embodiments, examples of the predetermined criteria can include but are not limited to: a channel congestion weighting factor, a number of connected clients weighting factor, a CPU weighting factor, an acceptance weight threshold.
- According to certain embodiments, a load balancer maintains a list (that is associated with a given requesting client) of access points for ranking (also referred to as a “ranking list’). According to certain embodiments, the load balancer ranks the ranking list of access points as a function of a channel congestion, a CPU usage and number of connected clients of a given AP.
- According to certain embodiments, the load balancer ranks APs in the ranking list in view of each AP's deviation from the worst case value for each parameter of a predetermined set of parameters.
-
FIG. 11 is a high-level network diagram showing aspects of a distributed load balancing solution, according to certain embodiments. InFIG. 11 , a client 1102 (also referred to as “a requesting client”) that wishes to connect to the best access point (or connect to a suitable access point) sends a request probe to each wireless access point that is nearby the requesting client, according to certain embodiments. Each access point that receives a probe request is also referred to as a “current AP”. - In
FIG. 11 , according to certain embodiments,current AP 1104 communicates with a clientload balance server 1106. Certain aspects ofcurrent AP 1104 are described in greater detail with reference to at leastFIG. 12 , herein. - In
FIG. 11 , according to certain embodiments, clientload balance server 1106 determines (1108) the distance of thecurrent AP 1104 from requestingclient 1102 based on the received signal strength indication (RSSI) of requestingclient 1102 as perceived bycurrent AP 1104. - In
FIG. 11 , according to certain embodiments, clientload balance server 1106 maintains a group of APs for each requesting client and determines if thecurrent AP 1104 is to be added (1110), based on the RSSI of the requestingclient 1102, to a “ranking list” associated with requestingclient 1102. Certain aspects of the clientload balance server 1106 are described in greater detail with reference to at leastFIG. 13 , herein. - In
FIG. 11 , according to certain embodiments, clientload balance server 1106 ranks (1112) the APs in the ranking list associated with requestingclient 1102 to determine the best AP (“BAP”). Certain aspects of the ranking process are described in greater detail with reference to at leastFIG. 13 ,FIG. 14 andFIG. 15 , herein. - In
FIG. 11 , according to certain embodiments, clientload balance server 1106 communicates (1114) information on a set of parameters of the BAP tocurrent AP 1104. - In
FIG. 11 , according to certain embodiments, a decision logic engine decides (1116) whether to send a response (1118) to the requestingclient 1102 or not to send a response (1120) to the requestingclient 1102. Certain aspects of such a decision process are described in greater detail with reference to at leastFIG. 16 , andFIG. 17 , herein. - According to certain embodiments, the process described with reference to
FIG. 11 occurs for each of the nearby APs that receive the request probe from the requestingclient 1102. -
FIG. 12 is a high-level flow chart that illustrates some of the functions of a current AP that receives a probe request, according to certain embodiments. Inblock 1202, the current AP receives a probe request from a requesting client. - At
block 1204, the current AP determines if the requesting client is a new client. If the current AP determines that the requesting client is a new client, then atblock 1206, the current AP sends a request to client load balance server (also referred to as a “CLS”) to find the best access point (BAP) for the requesting client. According to certain embodiments, along with the request to find BAP, the current AP sends at least a subset of the following information to the CLS: 1) RSSI of the requesting client as seen by the current AP, 2) current channel congestion (or channel utilization, “CU”) associated with the current AP, 3) current CPU utilization of the current AP, 4) the number of clients connected to the current AP, 5) media access control address (MAC address) of the current AP, and 6) media access control address (MAC address) of the requesting client. - At
block 1208, the current AP waits for a response from the CLS for up to a predetermined maximum silent period, according to certain embodiments. Atblock 1210, it is determined if the time for receiving a response from the CLS is less than the predetermined maximum silent period. - According to certain embodiments, the CLS determines the identity of the BAP and related information (BAP parameters) in response to the current AP's request referred to at
block 1206. According to certain embodiments, the determination of the identity of the BAP and related BAP parameters is described in greater detail with reference to at leastFIGS. 13 and 14 , herein. - If the time for receiving a response from the CLS is not less than the predetermined maximum silent period, then at
block 1214, the current AP sends a response to the requesting client. However, if the time for receiving a response from the CLS is less than the predetermined maximum silent period, then atblock 1212, the current AP waits for the information on the BAP and sends such information to a decision logic engine 1218. According to certain embodiments, the decision logic engine may be physically part of the current AP device. According to certain other embodiments, the decision logic engine may be remote from the current AP device. The manner in which the decision logic engine decides whether to respond to the requesting client is described in greater detail with respect to at leastFIG. 16 andFIG. 17 , herein. - At
block 1220, the current AP receives a decision from the decision logic engine and determines whether the decision logic engine has decided to accept or reject the requesting client. If it is determined that the requesting client should be accepted, the current AP sends a response to the requesting client atblock 1222. If it is determined that the requesting client is to be rejected, the current AP will not respond to the requesting client, according to certain embodiments. - According to certain embodiments, if at
block 1204, the current AP determines that the requesting client is not a new client, then atblock 1224, the current AP determines if the previous decision made by the decision logic engine is still valid based on how long ago the decision was made. For example, the previous decision remains valid if the age of the previous decision is less than a maximum decision age value. - If at
block 1224 the current AP determines that the previous decision made by the decision logic engine is still valid, then control passes to block 1220 andblock 1220 has been described above. - If at
block 1224 the current AP determines that the previous decision made by the decision logic engine is not valid, then atblock 1226, the current AP determines if the BAP is a known BAP. If the current AP determines that the BAP is not known, then control passes to block 1206 andblock 1206 has been previously described above. - If the current AP determines that the BAP is a known BAP, then at
block 1228, the current AP determines if the age of the BAP is less than a maximum age threshold. If the current AP determines that the age of the BAP is less than a maximum age threshold, then control passes to block 1218 so that the decision logic engine can determine whether to accept or reject the requesting client as previously described above. If however, the current AP determines that the age of the BAP is greater than the maximum age threshold, then control passes to block 1206 andblock 1206 has been previously described above. -
FIG. 13 is a high-level flow chart that illustrates some of the operations of the client load balance server (CLS). Atblock 1302, the CLS receives, from the current AP, the request to find the best access point (BAP) for the requesting client, as previously described with reference to at leastFIG. 12 above. According to certain embodiments, along with the request to find the BAP, the CLS receives the following information from the current AP: 1) RSSI of the requesting client as seen by the current AP, 2) current channel congestion (or channel utilization percentage, “CU”) associated with the current AP, 3) current CPU utilization percentage of the current AP, 4) the number of clients connected to the current AP, 5) media access control address (MAC address) of the current AP, and 6) media access control address (MAC address) of the requesting client. - According to certain embodiments, the CLS associates a group of APs with each client that is known to the CLS. Thus, when the CLS receives, from the current AP, the request to find the best access point (BAP) for the requesting client, the CLS adds, at
block 1304, the current AP to the group of APs associated with the requesting client if the requesting client is previously known to the CLS. Further, atblock 1304, if the requesting client is not previously known to the CLS, then the CLS creates a new group of APs for the requesting client, according to certain embodiments. - At
block 1306, the CLS determines the distance of the requesting client from the current AP. As a non-limiting example, the distance can be determined using triangulation techniques. - At
block 1308, the CLS determines if the distance of the requesting client from the current AP is less than a predetermined distance threshold. If the distance is less than the predetermined distance threshold, then atblock 1312 the CLS adds the current AP to a ranking list associated with the requesting client. However, if the distance is not less than the predetermined distance threshold, then atblock 1310, the CLS omits the current AP from the ranking list associated with the requesting client, according to certain embodiments. - At block 1314, the CLS determines if the number of APs in the ranking list is greater than 1. If the number of APs in the ranking list is greater than 1, then at
block 1316, the CLS ranks the AP in the list. The manner of ranking is described in greater detail with reference to at leastFIG. 14 andFIG. 15 , herein, according to certain embodiments. Such a ranking identifies the best AP (BAP). Atblock 1318, the CLS sends information of the BAP, such as the identity of the BAP and other BAP parameters, to the current AP. - According to certain embodiments, the BAP information includes at least a subset of: 1) RSSI of the requesting client as seen by BAP, 2) channel utilization percentage associated with BAP, 3) BAP CPU utilization percentage, 4) the number of clients connected to BAP, 5) media access control address (MAC address) of BAP, and 6) media access control address (MAC address) of the requesting client.
- If at block 1314, the CLS determines that the number of APs in the ranking list is not greater than 1, then at
block 1320, the CLS waits for more APs to make request for BAPs. According to certain embodiments, the CLS waits for a period up to a predetermined maximum silent period. - At
block 1322, the CLS determines if the predetermined maximum silent period has expired. If the predetermined maximum silent period has expired then atblock 1324, the CLS designates the current AP as BAP and sends the BAP information to the current AP atblock 1318 as previously described. - If the predetermined maximum silent period has not expired then controls passes to block 1316 and
block 1316 is previously described above. -
FIG. 14 is a high-level flow chart that illustrates some of the features of the ranking operation for ranking the APs, according to certain embodiments. The higher the AP's rank, the better the AP. Atblock 1402, the CLS considers each AP in the ranking list in view of each AP's deviation from the worst case value for each parameter of a predetermined set of parameters, as described with reference toblocks FIG. 14 . - At
block 1404, for each current AP, the CLS determines the absolute value of the difference between channel utilization maximum threshold value and current AP channel utilization. - At
block 1406, for each current AP, the CLS determines the absolute value of the difference between CPU usage max threshold value and current AP CPU usage adds it to the result ofblock 1404. - At
block 1408, for each current AP, the CLS determines the absolute value of the difference between number of clients connected to the best AP and the number clients connected to the current AP and adds it to the result ofblock 1406. According to certain embodiments, the result ofblock 1408 is the rank of the respective current AP. - At
block 1410, the CLS determines if the current AP has a rank that is greater than the rank of the best AP thus far. If the CLS determines that the current AP's rank is greater than the best rank, then atblock 1414, the CLS designates the current AP as the BAP. Next control passes to block 1412 which is described below. - If at
block 1410, the CLS determines that the current AP's rank is not greater than the rank of the best AP thus far, then atblock 1412, the CLS determines if there are more APs in the rank list. If there are no more APs in the rank list, then atblock 1416, the CLS sends the BAP information to the current AP. If there are more APs in the rank list then control passes back to block 1402 to determine the rank of the next current AP. - According to certain embodiments, the APs in the ranking list are ranked as follows based on empirical analysis:
-
Rank=(Δx/10)+((2+(K/3)*2)*CUcurr/CUmax)*(Δx/10)*(Ŝ)+(Δy/10+((2+(L/3)*2)*CPUcurr/CPUmax)+Δy/10)*(Ŝ)+(RSSI/30)+(Δz)*(Ŝ), for all K>0 and L>0 - and where,
-
(Δx)=ABS(80−CUrap) - Absolute value of the difference between channel utilization maximum threshold value and current AP channel utilization.
-
(Δy)=ABS(80−CPUrap) - Absolute value of the difference between CPU utilization max threshold value and current AP CPU utilization
-
(Δz)=ABS(NCbest−NCrap) - Absolute value of the difference between number of clients connected to the best AP and the number clients connected to the current AP.
- And where, the sign factor (Ŝ):
- (Ŝ)=((comparing value−current value)>0), if current value is greater than the comparing value, the sign factor is true. If the sign factor is true, the calculated value will be negative. For example, if the current AP channel utilization is 83 and CUmax is 80, the sign factor will be true and will pull the results toward the negative scale.
-
K=(CUcurr−CUmax) -
L=(CPUcurr−CPUmax) -
FIG. 15 is a graph that illustrates aspects of the ranking logic used for load balancing clients across APs in a wireless environment, according to certain embodiments. Rankinglogic graph 1500 ofFIG. 15 shows Rank 1502 as the vertical axis and Rank vs channel utilization/CPU utilization (CU/CPU) 1504 as the horizontal axis.Graph 1500 also shows the rank vsCPU plot 1506 and the rank vsCU plot 1508.Graph 1500 also shows theworst case value 1510 based on the example above. To explain, the current APs are ranked based on the respective current AP's deviation from the worst case value for each parameter of a predetermined set of parameters, according to certain embodiments. As non-limiting examples, the parameters include channel utilization, CPU utilization and number of connected clients, according to certain embodiments. -
FIG. 16 is a high-level flow chart that illustrates some aspects of the decision logic engine used by the current AP (when the current AP receives the BAP information from the CLS) to decide whether to respond or ignore the requesting client that is requesting connection with the current AP. - According to certain embodiments, such a decision of the current AP can be any one of the following types: 1) aggressive, 2) inclined, and 3) fair. According to certain embodiments, when an aggressive decision is selected, the current AP responds immediately to the requesting client without further delay or further processing. When an inclined decision is selected, the current AP processes certain parameters as described in greater detail below. When a fair decision is selected, the current AP will attempt to divide the number of clients fairly across the current AP and the BAP.
- At
block 1602, the decision logic engine determines a channel utilization (CU) weight between BAP and the current AP. Atblock 1604, the decision logic engine determines if the weight determined atblock 1602 is greater than a predetermined maximum aggressive weight threshold value. If the weight determined atblock 1602 is greater than the maximum aggressive weight threshold value then, atblock 1608, the decision logic engine determines if the weight determined atblock 1602 is greater than or equal to a predetermined “accept weight” threshold value. If the weight determined atblock 1602 is greater than or equal to the predetermined “accept” threshold value, then atblock 1610 the decision status is set to “accept” (so that the current AP can send a response to the requesting client to allow the requesting client to connect to the current AP. However, if the weight determined atblock 1602 is not greater than or equal to the predetermined “accept” threshold value, then at block 1612 the decision status is set to “reject” (so that the current AP can ignore the requesting client probe request). - Further, if at
block 1604, the decision logic engine determines that the weight determined atblock 1602 is not greater than the predetermined maximum aggressive weight threshold value, then at block 1606, the decision logic engine determines the CPU weight between the BAP and the current AP. - At
block 1614, the decision logic engine adds the CPU weight to the CU weight, according to certain embodiments. Atblock 1616, the decision logic engine determines if the combined CPU and CU weight is greater than the maximum inclined weight threshold value. If the decision logic engine determines that the combined CPU and CU weight is greater than the maximum inclined weight threshold value, then control passes to block 1608. However, if combined CPU and CU weight is not greater than the maximum inclined weight threshold value, then atblock 1618, the decision logic engine determines the “number-of-connected-clients” weight between the BAP and the current AP (this is the fair decision). Then atblock 1608, the decision logic engine determines if the weight determined atblock 1618 is greater than or equal to the predetermined “accept” threshold value. If the weight determined atblock 1618 is greater than or equal to a predetermined “accept” threshold value, then atblock 1610 the decision status is set to “accept” (so that the current AP can send a response to the requesting client to allow the requesting client to connect to the current AP. However, if the weight determined atblock 1618 is not greater than or equal to the predetermined “accept” threshold value, then at block 1612 the decision status is set to “reject” (so that the current AP can ignore the requesting client probe request). - According to certain embodiments, the weight determination for each of the parameters (CU, CPU and Number of connected clients) is as follows and is based on empirical analysis.
- CU Weight Determination
-
Weight=P+Weight+((Δx/10+((2+(K/3)*2)*b/m))+(((2+(L/3)*2)*a/m)+Δx/10))*(Ŝ) - for all a>i∥b>i∥Δx>20 and for all K>0 and L>0
- The Weight will be non-zero if and only if a or b are greater than the Minimum channel utilization threshold I, (for example, i=30) or Δx>20, according to certain embodiments.
- where,
- P=Parameter priority
- P can be either negative or positive or zero. P can be used either to increase or decrease the priority of a given parameter. The higher the value of P, the higher the priority. Default value of P is 1. The value P can be changed by the user. The value of P is unique for each parameter, according to certain embodiments.
- m=Maximum threshold value of the CU
- a=Current AP channel utilization
- b=Best AP channel utilization
- i=Minimum CU threshold
- K=(b−m)
- L=(a−m)
- Δx=ABS (a−b), which is the absolute value of the difference between current AP channel utilization and Best AP channel utilization.
- (Ŝ)−Sign factor=((b−a)>0), if current CU value is greater than the Best AP CU value the sign factor will be true. If the sign factor is true the calculated value will be negative. For example, if Current AP CU is 33 and Best AP CU is 30, the sign factor will be true.
- CPU Weight Determination
-
Weight=P+Weight+((Δx/10+((2+(K/3)*2)*b/m))+(((2+(L/3)*2)*a/m)+Δx/10))*(Ŝ) - for all a>i∥b>i∥Δx>20 and for all K>0 and L>0
- The input weight will be always a non-zero value in this equation.
- P=Parameter priority
- P can be either negative or positive or zero. P can be used either to increase or decrease the priority of a given parameter. The higher the value of P, the higher the priority. Default value of P is 1. The value P can be changed by the user. The value of P is unique for each parameter, according to certain embodiments.
- where,
- m=Max threshold value of the CPU utilization
- a=Current AP CPU utilization
- b=Best AP CPU utilization
- i=minimum CPU threshold
- P=Priority
- Δx=ABS (a−b) (Absolute difference between current AP CPU utilization and Best AP CPU utilization).
- K=(b−m)
- L=(a−m)
- (Ŝ)−Sign factor=((b−a)>0), if current CPU utilization value is greater than the best AP CPU utilization value, then the sign factor will be true. If the sign factor is true, the calculated value will be negative. For example if Current AP CPU is 33 and Best AP CPU is 30, the sign factor will be true.
- Number of Connected Clients Weight Determination
-
Weight=P+Weight+Δx/S*(Ŝ)+(ΔRSSI/30) -
- The scaling parameter (S) value will change based on the weight calculated from the previous equations used by parameters CU and CPU.
- If the input weight<3
- S=1
- If the input weight>3
- S=2
- If the input weight>=5
- S=3
- P=Parameter priority
- P can be either negative or positive or zero. P can be used either to increase or decrease the priority of a given parameter. The higher the value of P, the higher the priority. Default value of P is 1. The value P can be changed by the user. The value of P is unique for each parameter, according to certain embodiments.
- where,
- S=Scale
- a=Current AP number of connected clients
- b=Best AP number of connected clients
- P=Priority
- Δx=ABS (a−b) (Absolute difference between current AP connected clients and Best AP connected clients)
- ΔRSSI=(a−b) (A positive difference between current AP RSSI and Best AP RSSI)
- (Ŝ)−Sign factor=((b−a)>0), if current AP number of connected clients value is greater than the best AP number of connected clients value, then the sign factor will be true. If the sign factor is true, the calculated value will be negative. For example if Current AP number of connected clients is 33 and Best AP number of connected clients is 30, the sign factor will be true.
-
FIG. 17 is a graph that illustrates aspects of the decision logic used for load balancing clients across APs in a wireless environment, according to certain embodiments.Decision logic graph 1700 ofFIG. 17 shows Weights 1702 as the vertical axis and BAP (x) vs Curr AP (x=0) 1704 as the horizontal axis.Graph 1700 also shows the BAP (x) vs Curr AP (x=0)plot 1706 and the Curr AP vs BAP (x=0)plot 1708.Graph 1500 also shows the worst case.Graph 1700 also shows theFair 1710, Inclined 1712, and Aggressive 1714 decision logic. - The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.
Claims (9)
1. A wireless access point device comprising:
a dedicated network processor;
at least one application container; and
an application manager application that monitors one or more third party applications after the one or more third party applications are installed in the at least one application container;
wherein the at least one application container includes a mobile application services engine application programming interface (API).
2. The wireless access point device of claim 1 , further comprising a container file system for use by the one or more third party applications in the application container, wherein the container file system is confined to a pre-allocated segregated disk space within a file system of the wireless access point device.
3. The wireless access point device of claim 1 , further comprising a plurality of application services that can be accessed by the one or more third party applications in the application container through the mobile application services engine API.
4. The wireless access point device of claim 3 , wherein the plurality of application services that can be accessed by the one or more third party applications in the application container through the mobile application services engine API include Hyper Text Transfer Protocol (HTTP) proxy services.
5. The wireless access point device of claim 3 , wherein the plurality of application services that can be accessed by the one or more third party applications in the application container through the mobile application services engine API include fast common gateway interface (FCGI) web hosting services.
6. The wireless access point device of claim 3 , wherein the plurality of application services that can be accessed by the one or more third party applications in the application container through the mobile application services engine API include deep packet inspection (DPI) services.
7. The wireless access point device of claim 2 , wherein the container file system prevents the one or more third party applications from accessing disk space outside the container file system.
8. The wireless access point device of claim 1 , further comprising a wireless local area network (WLAN) chipset capable of connecting more than fifty wireless client connections to a wired network at the same time.
9. The wireless access point device of claim 1 , further comprising a stand-alone multi-core central processing unit (CPU).
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/789,904 US20180300190A1 (en) | 2015-06-24 | 2017-10-20 | Mobile application service engine (mase) |
US16/654,836 US11157340B2 (en) | 2015-06-24 | 2019-10-16 | Mobile application service engine (MASE) |
US17/510,233 US20220043701A1 (en) | 2015-06-24 | 2021-10-25 | Mobile application service engine (mase) |
US17/576,630 US20220138029A1 (en) | 2015-06-24 | 2022-01-14 | Distributed load balancing for access points |
US17/867,517 US11720428B2 (en) | 2015-06-24 | 2022-07-18 | Mobile application service engine (MASE) |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/749,580 US20160381597A1 (en) | 2015-06-24 | 2015-06-24 | WiFi Airtime Allocation |
US14/886,636 US20170111821A1 (en) | 2015-10-19 | 2015-10-19 | Distributed load balancing for access points |
US14/938,763 US20170131987A1 (en) | 2015-11-11 | 2015-11-11 | Mobile Application Service Engine (MASE) |
US15/789,904 US20180300190A1 (en) | 2015-06-24 | 2017-10-20 | Mobile application service engine (mase) |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/938,763 Continuation-In-Part US20170131987A1 (en) | 2015-06-24 | 2015-11-11 | Mobile Application Service Engine (MASE) |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/654,836 Continuation US11157340B2 (en) | 2015-06-24 | 2019-10-16 | Mobile application service engine (MASE) |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180300190A1 true US20180300190A1 (en) | 2018-10-18 |
Family
ID=63790061
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/789,904 Abandoned US20180300190A1 (en) | 2015-06-24 | 2017-10-20 | Mobile application service engine (mase) |
US16/654,836 Active US11157340B2 (en) | 2015-06-24 | 2019-10-16 | Mobile application service engine (MASE) |
US17/510,233 Pending US20220043701A1 (en) | 2015-06-24 | 2021-10-25 | Mobile application service engine (mase) |
US17/576,630 Pending US20220138029A1 (en) | 2015-06-24 | 2022-01-14 | Distributed load balancing for access points |
US17/867,517 Active US11720428B2 (en) | 2015-06-24 | 2022-07-18 | Mobile application service engine (MASE) |
Family Applications After (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/654,836 Active US11157340B2 (en) | 2015-06-24 | 2019-10-16 | Mobile application service engine (MASE) |
US17/510,233 Pending US20220043701A1 (en) | 2015-06-24 | 2021-10-25 | Mobile application service engine (mase) |
US17/576,630 Pending US20220138029A1 (en) | 2015-06-24 | 2022-01-14 | Distributed load balancing for access points |
US17/867,517 Active US11720428B2 (en) | 2015-06-24 | 2022-07-18 | Mobile application service engine (MASE) |
Country Status (1)
Country | Link |
---|---|
US (5) | US20180300190A1 (en) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070282951A1 (en) * | 2006-02-10 | 2007-12-06 | Selimis Nikolas A | Cross-domain solution (CDS) collaborate-access-browse (CAB) and assured file transfer (AFT) |
US20110296046A1 (en) * | 2010-05-28 | 2011-12-01 | Ortiva Wireless, Inc. | Adaptive progressive download |
US20140109072A1 (en) * | 2012-10-16 | 2014-04-17 | Citrix Systems, Inc. | Application wrapping for application management framework |
US20140148129A1 (en) * | 2011-06-30 | 2014-05-29 | Telefonaktiebolaget L M Ericsson (Publ) | WiFi FIXED WIRELESS PERSONAL SERVICES |
US20140317276A1 (en) * | 2013-03-15 | 2014-10-23 | Aerohive Networks, Inc. | Application based data traffic routing using network tunneling |
US20160205539A1 (en) * | 2013-07-01 | 2016-07-14 | Nokia Technologies Oy | Configurable base station |
US20160241702A1 (en) * | 2014-05-29 | 2016-08-18 | Fortinet, Inc. | Management of wireless access points via virtualization |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2861246B1 (en) | 2003-10-17 | 2006-01-27 | Cit Alcatel | WIRELESS COMMUNICATIONS NETWORK WITH RADIO ACCESS POINTS AT DYNAMICALLY ADAPTABLE TRANSMISSION POWER LEVEL |
US7450904B2 (en) | 2004-01-08 | 2008-11-11 | Interdigital Technology Corporation | Escape mechanism for a wireless local area network |
US7610057B2 (en) | 2004-04-23 | 2009-10-27 | Microsoft Corporation | Selecting a wireless networking technology on a device capable of carrying out wireless network communications via multiple wireless technologies |
US7480264B1 (en) | 2005-02-10 | 2009-01-20 | Sonicwall, Inc. | Centralized wireless LAN load balancing |
US7554962B2 (en) | 2006-03-20 | 2009-06-30 | Nokia Corporation | Method, mobile station, and software product for access point selection |
US7729278B2 (en) | 2007-02-14 | 2010-06-01 | Tropos Networks, Inc. | Wireless routing based on data packet classifications |
US7957337B2 (en) | 2007-03-30 | 2011-06-07 | Tropos Networks, Inc. | Air-time control of wireless networks |
US8559407B2 (en) | 2007-08-31 | 2013-10-15 | Tropos Networks, Inc. | Characterizing uncoordinated interference of a wireless network |
US8259635B2 (en) | 2008-01-04 | 2012-09-04 | Cisco Technology, Inc. | Automatic clustering of wireless network nodes toward selected mesh access points |
CN101971666B (en) | 2008-01-14 | 2014-06-25 | 发尔泰公司 | Service differentiation and service level agreements for wireless access clients |
GB2490607B (en) | 2008-01-24 | 2013-03-20 | Firetide Inc | Channel assignment for wireless access networks |
US8559306B2 (en) | 2008-02-13 | 2013-10-15 | Cisco Technology, Inc. | End-to-end packet aggregation in mesh networks |
US8155058B2 (en) | 2009-01-30 | 2012-04-10 | Aruba Networks, Inc. | Client balancing in wireless networks |
US8638731B2 (en) | 2009-07-06 | 2014-01-28 | Intel Corporation | Femtocell architecture and network |
US9124409B2 (en) | 2009-07-30 | 2015-09-01 | Qualcomm Incorporated | Determining control region parameters for multiple transmission points |
US8942717B2 (en) | 2009-11-30 | 2015-01-27 | Intel Corporation | Load balancing techniques in wireless networks |
US8826413B2 (en) | 2009-12-30 | 2014-09-02 | Motorla Solutions, Inc. | Wireless local area network infrastructure devices having improved firewall features |
US8873440B2 (en) * | 2010-03-29 | 2014-10-28 | Qualcomm Incorporated | Maintaining different virtual active sets for different cell types |
TWI415501B (en) * | 2010-07-20 | 2013-11-11 | Gemtek Technology Co Ltd | Wireless network system and wireless access point device thereof |
JP5410410B2 (en) * | 2010-12-24 | 2014-02-05 | 株式会社Nttドコモ | Base station, radio communication system, and communication method |
US8660008B2 (en) * | 2011-03-02 | 2014-02-25 | 3Inova Networks Inc. | Traffic management in distributed wireless networks |
US8767680B2 (en) | 2011-07-21 | 2014-07-01 | Moxa Inc. | Roaming system using wireless access controller to select access point and method thereof |
US9326313B2 (en) | 2011-08-01 | 2016-04-26 | Aruba Networks, Inc. | System, apparatus and method for managing client devices within a wireless network |
WO2013047952A1 (en) | 2011-09-26 | 2013-04-04 | 엘지전자 주식회사 | Method and apparatus for allocating minimum guaranteed resource amount to access point in wireless access system |
US9173111B2 (en) | 2011-12-28 | 2015-10-27 | Silver Spring Networks, Inc. | System and method for convergence and automatic disabling of access points in a wireless mesh network |
US20130267257A1 (en) | 2012-04-10 | 2013-10-10 | Qualcomm Incorporated | Access point measurements for received signal prediction |
US8902777B1 (en) | 2012-06-29 | 2014-12-02 | Juniper Networks, Inc. | Methods and apparatus for self-tuning aggregation of data units |
US9277532B2 (en) * | 2012-09-18 | 2016-03-01 | Alcatel Lucent | Methods for allocating and scheduling uplink and downlink transmissions and apparatuses thereof |
EP2785103A1 (en) | 2013-03-28 | 2014-10-01 | British Telecommunications public limited company | Method and system for controlling traffic in a wireless lan |
EP2785093A1 (en) | 2013-03-28 | 2014-10-01 | British Telecommunications public limited company | Access point selection in a wireless network |
US9414301B2 (en) | 2013-04-26 | 2016-08-09 | Telefonaktiebolaget Lm Ericsson (Publ) | Network access selection between access networks |
US20140334336A1 (en) | 2013-05-10 | 2014-11-13 | Relay2, Inc. | Multi-Tenant Virtual Access Point- Network Resources Virtualization |
US9699062B2 (en) | 2013-05-31 | 2017-07-04 | Telecom Italia S.P.A. | Performance measurement of a link of a packet-switched communication network |
US9225602B2 (en) | 2013-07-30 | 2015-12-29 | Aruba Networks, Inc. | Dynamic grouping and configuration of access points |
US20150045022A1 (en) | 2013-08-06 | 2015-02-12 | Gaby Prechner | Access points and methods for access point selection using an information data structure |
KR102091138B1 (en) | 2013-09-12 | 2020-03-19 | 삼성전자주식회사 | Method for Data Transmission for Wireless Network Environment and Data Transmitter |
WO2015041663A1 (en) | 2013-09-20 | 2015-03-26 | Intel Corporation | Selective utilization of consumer shared access points to facilitate optimized wireless communications |
US9763117B2 (en) | 2014-03-24 | 2017-09-12 | Netgear, Inc. | Multi-band wireless station having multiple radios in one band |
US9510233B2 (en) | 2014-07-15 | 2016-11-29 | Aruba Networks, Inc. | Intelligent handling of voice calls from mobile voice client devices |
US9503344B2 (en) | 2014-07-25 | 2016-11-22 | Telefonaktiebolaget L M Ericsson (Publ) | Data path performance measurement using network traffic in a software defined network |
US9571249B2 (en) | 2014-07-29 | 2017-02-14 | Aruba Networks, Inc. | Dynamic control of channel sounding for transmit beamforming |
US20160112944A1 (en) * | 2014-10-17 | 2016-04-21 | Qualcomm Incorporated | Access point selection based on association performance |
US9538446B1 (en) | 2015-07-29 | 2017-01-03 | Fortinet, Inc | Directed station roaming in cloud managed Wi-Fi network |
US9521648B1 (en) | 2015-06-26 | 2016-12-13 | Intel Corporation | Location estimation and wireless display device connection method and device |
-
2017
- 2017-10-20 US US15/789,904 patent/US20180300190A1/en not_active Abandoned
-
2019
- 2019-10-16 US US16/654,836 patent/US11157340B2/en active Active
-
2021
- 2021-10-25 US US17/510,233 patent/US20220043701A1/en active Pending
-
2022
- 2022-01-14 US US17/576,630 patent/US20220138029A1/en active Pending
- 2022-07-18 US US17/867,517 patent/US11720428B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070282951A1 (en) * | 2006-02-10 | 2007-12-06 | Selimis Nikolas A | Cross-domain solution (CDS) collaborate-access-browse (CAB) and assured file transfer (AFT) |
US20110296046A1 (en) * | 2010-05-28 | 2011-12-01 | Ortiva Wireless, Inc. | Adaptive progressive download |
US20140148129A1 (en) * | 2011-06-30 | 2014-05-29 | Telefonaktiebolaget L M Ericsson (Publ) | WiFi FIXED WIRELESS PERSONAL SERVICES |
US20140109072A1 (en) * | 2012-10-16 | 2014-04-17 | Citrix Systems, Inc. | Application wrapping for application management framework |
US20140317276A1 (en) * | 2013-03-15 | 2014-10-23 | Aerohive Networks, Inc. | Application based data traffic routing using network tunneling |
US20160205539A1 (en) * | 2013-07-01 | 2016-07-14 | Nokia Technologies Oy | Configurable base station |
US20160241702A1 (en) * | 2014-05-29 | 2016-08-18 | Fortinet, Inc. | Management of wireless access points via virtualization |
Also Published As
Publication number | Publication date |
---|---|
US20220043701A1 (en) | 2022-02-10 |
US11720428B2 (en) | 2023-08-08 |
US20200285529A1 (en) | 2020-09-10 |
US20220138029A1 (en) | 2022-05-05 |
US20220350688A1 (en) | 2022-11-03 |
US11157340B2 (en) | 2021-10-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3646572B1 (en) | Methods and systems for network slicing | |
US20150319081A1 (en) | Method and apparatus for optimized network and service processing | |
CN110463140B (en) | Network service level agreement for computer data center | |
US9680708B2 (en) | Method and apparatus for cloud resource delivery | |
KR20180014060A (en) | System and method for providing and distributing spectrum resources | |
JP2020502948A (en) | Packet transmission system and method | |
US20150319050A1 (en) | Method and apparatus for a fully automated engine that ensures performance, service availability, system availability, health monitoring with intelligent dynamic resource scheduling and live migration capabilities | |
US20150304281A1 (en) | Method and apparatus for application and l4-l7 protocol aware dynamic network access control, threat management and optimizations in sdn based networks | |
EP3286966B1 (en) | Resource reallocation | |
WO2017031698A1 (en) | Method, apparatus, and system for acquiring vnf information | |
US20150263906A1 (en) | Method and apparatus for ensuring application and network service performance in an automated manner | |
US20150281006A1 (en) | Method and apparatus distributed multi- cloud resident elastic analytics engine | |
US20150263885A1 (en) | Method and apparatus for automatic enablement of network services for enterprises | |
US20170131987A1 (en) | Mobile Application Service Engine (MASE) | |
US20150263980A1 (en) | Method and apparatus for rapid instance deployment on a cloud using a multi-cloud controller | |
KR102033402B1 (en) | Smart gateway supporting iot and realtime traffic shaping method for the same | |
Xing et al. | A highly scalable bandwidth estimation of commercial hotspot access points | |
Caraguay et al. | Framework for optimized multimedia routing over software defined networks | |
US11720428B2 (en) | Mobile application service engine (MASE) | |
CN113169997B (en) | Method, apparatus, and storage medium for providing variable-sized file based on device and network conditions | |
Ben Mustafa et al. | FlexStream: Towards flexible adaptive video streaming on end devices using extreme SDN | |
US20150281005A1 (en) | Smart network and service elements | |
WO2023057794A1 (en) | Method for aligning quality of service in mobile network and edge cloud | |
Diarra et al. | RAPID: A RAN-aware performance enhancing proxy for high throughput low delay flows in MEC-enabled cellular networks | |
Ganji | Network Performance Analysis of IoT, Cloud and Mobile Systems |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |