CN110113221B

CN110113221B - System and method for dynamic bandwidth adjustment of cellular interfaces in a network environment

Info

Publication number: CN110113221B
Application number: CN201910541464.6A
Authority: CN
Inventors: 苏杰·戈皮纳坦; 杰士林·安东尼·普森帕兰比; 斯瓦普纳·阿南达; 比诺德·罗伊; 维韦克·达塔尔
Original assignee: Cisco Technology Inc
Current assignee: Cisco Technology Inc
Priority date: 2015-03-05
Filing date: 2015-09-06
Publication date: 2022-05-06
Anticipated expiration: 2035-09-06
Also published as: CN110113221A

Abstract

The present disclosure relates to systems and methods for dynamic bandwidth adjustment of cellular interfaces in a network environment. A method is provided in one example embodiment and may include determining a predicted average throughput for each of one or more cellular interfaces and adjusting a bandwidth of each of the one or more cellular interfaces based at least in part on the determined predicted average throughput for each of the one or more cellular interfaces. Another method can be provided that can include determining a variance of path metrics for a plurality of cellular interfaces and updating a routing table of a cellular interface using the determined variance if there is a difference between the determined variance and a previous variance determined for the cellular interface. Another method can be provided, which can include: monitoring a water level flag threshold of the MAC buffer; generating an interrupt when a specific water level marker threshold of the MAC is reached; and adjusting enqueuing of the uplink packet into the MAC buffer based on the interruption.

Description

System and method for dynamic bandwidth adjustment of cellular interface in network environment

The present application is a divisional application entitled "system and method for dynamic bandwidth adjustment of a cellular interface in a network environment" filed on date 2015, 09/06, application number 201510561240.3.

Related applicationPlease note that

This application is a continuation of the application serial No. 14/639,748 (and claims priority benefits in accordance with u.s.c. § 120) filed 3, 5/2015, entitled "system and METHOD FOR DYNAMIC BANDWIDTH adjustment of CELLULAR interfaces in a NETWORK ENVIRONMENT" by Sujay gopinat athan et al. The disclosure of the prior application is considered part of (and is incorporated by reference in its entirety into) the disclosure of the present application.

Technical Field

The present disclosure relates generally to the field of communications, and more particularly, to systems and methods for dynamic bandwidth adjustment of cellular interfaces in a network environment.

Background

Networking architectures in communication environments, particularly mobile wireless environments, have become more and more complex. Mobile communication networks have gained significant development as user equipment is increasingly connected to mobile radio environments. In some instances, the user device can include a router that can provide routing traffic through a network using wired, wireless, and/or cellular interfaces. The flow of traffic across the cellular interface can vary significantly, e.g., with environmental conditions, cellular interface, number of users, etc. Since traffic routing is often time critical, delayed flows over the cellular interface can result in system congestion, lost packets, and/or overloaded router resources, among other things. Thus, there are significant challenges in managing the cellular interfaces of routing devices in a network environment.

Disclosure of Invention

A method is provided in one example embodiment and may include determining a predicted average throughput for each of one or more cellular interfaces of a router and adjusting a bandwidth of each of the one or more cellular interfaces based at least in part on the predicted average throughput determined for each of the one or more cellular interfaces of the router. In some examples, at least one of the cellular interfaces may be capable of connecting to an evolved node b (enodeb) of a Long Term Evolution (LTE) network.

In some examples, determining the predicted average throughput for the particular cellular interface can include determining one or more per resource block throughput values for the particular cellular interface, wherein each of the one or more per resource block throughput values is associated with a resource block associated with a particular uplink grant received by the particular cellular interface; determining one or more instantaneous throughput values for a particular cellular interface based on each of the one or more per-resource-block throughput values and a number of resource blocks to be transmitted for each of one or more uplink grants received by the cellular interface; and determining a predicted average throughput for the particular cellular interface based on a sum of one or more instantaneous throughput values related to a number of the one or more uplink grants received by the cellular interface.

In some instances, determining a per resource block throughput value for a particular resource block may be based on: a number of symbols to be used for transmitting the particular resource block, a number of bits per symbol of each symbol of the particular resource block, a number of subcarriers comprised by the resource block to be transmitted in the particular time frame, wherein the number of bits per symbol is based on a modulation coding scheme of the particular resource block.

In some cases, the method can include repeating the determining and adjusting steps at an update rate, wherein the update rate is configured based at least in part on whether the router is deployed in a fixed environment or a mobile environment. In other cases, the method can include determining an adjusted bandwidth for a particular cellular interface, comparing the adjusted bandwidth for the particular cellular interface to at least one quality of service (QoS) policy map configured for the cellular interface, wherein the QoS policy map comprises a plurality of QoS service policies each associated with a particular bandwidth threshold; and associating the particular QoS service policy to the particular cellular interface based on the comparison. In some instances, a particular QoS service policy can include an allocation of bandwidth to multiple traffic levels.

Drawings

A more complete understanding of the present disclosure and features and advantages thereof may be acquired by referring to the following description taken in conjunction with the accompanying drawings, in which like reference numbers indicate like parts, and wherein:

fig. 1A is a simplified block diagram illustrating communications including a router to facilitate dynamic bandwidth adjustment for a cellular interface in a network environment, according to one embodiment of the present disclosure;

FIG. 1B is a simplified block diagram illustrating details associated with one possible embodiment of an I/O interface of a router;

fig. 2 is a simplified flowchart illustrating example operations associated with providing dynamic bandwidth adjustment to a cellular interface in accordance with one possible embodiment of a router;

FIG. 3 is a simplified flowchart illustrating example operations associated with determining a predicted average throughput for a particular cellular interface of a router;

fig. 4 is a simplified flowchart illustrating example operations associated with providing adaptive quality of service for a cellular interface, according to one possible embodiment of a router;

FIG. 5 is a simplified block diagram illustrating additional details associated with one possible embodiment of a router;

fig. 6 is a simplified flowchart illustrating example operations associated with providing dynamic load balancing of a cellular interface, according to one possible embodiment of a router;

7A-7B are simplified block diagrams illustrating additional details associated with one possible embodiment of a router; and

fig. 8 is a simplified flowchart illustrating example operations associated with reducing packet delay during congestion in a cellular modem in accordance with one possible embodiment of a router.

Detailed Description

Turning to fig. 1A, fig. 1A is a simplified block diagram illustrating a communication system 100, the communication system 100 including a router 110 to facilitate providing bandwidth management for a cellular interface, according to one embodiment of the present disclosure. The example architecture of fig. 1A includes a router 110, the router 110 capable of providing multiple links to various elements or nodes of a network 130, including: one or more cellular links to evolved node bs (enodebs) 132, one or more wireless links to wireless local area network access points (WLAN APs) 136, and one or more wired links to wired network nodes 138. In various embodiments, router 110 can include host router interconnect 112, processor 114, memory element 116, host router logic 118, host router storage 120, and one or more I/O interfaces 124.1-124. N. The router 110 can also be referred to as an Integrated Services Router (ISR). As discussed in more detail below, fig. 1B is a simplified block diagram illustrating details associated with one possible embodiment of an I/O interface (e.g., I/O interface 124.1) of router 110.

In various embodiments, network 130 can represent a series of points or nodes of interconnected communication paths (wired or wireless) for receiving and transmitting packets of information that propagate through communication system 100. In various embodiments, the network 130 can be associated with and/or provided by a single network operator or service provider and/or multiple network operators or service providers. In various embodiments, the network 130 can include and/or overlap in whole or in part with one or more packet data networks (e.g., the internet). Network 130 may provide a communicative interface between selected nodes or elements of the network and router 110, and may be any Local Area Network (LAN), Wireless Local Area Network (WLAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), Virtual Private Network (VPN), Radio Access Network (RAN), enterprise network, intranet, extranet, or any other suitable structure or system that facilitates communications in a network environment.

In various embodiments, the network 130 may implement a User Datagram Protocol (UDP)/internet protocol (UDP/IP) connection and, in particular embodiments of the present disclosure, use the transmission control protocol (TCP/IP) communication language protocol. However, the communication network may alternatively implement any other suitable communication protocol for transmitting and receiving data within communication system 100. In various embodiments, the network 130 can be a plurality of networks interconnected by one or more network applications.

In general, the router 110 may facilitate the transfer of data and information between various elements of the network 130 using various links (e.g., cellular, wireless, wired) that can be provided via one or more I/O interfaces 124.1-124. N, which one or more I/O interfaces 124.1-124. N can be configured for the router 110 according to operator requirements. Although not shown, in various embodiments, router 110 can be included within network 130. As referred to herein in this specification, a "link" may refer to any communication path (wired or wireless) between an element or node of network 130 and a given I/O interface. In various embodiments, router 110 can be one or more of the following: network appliances, servers, switches, gateways, bridges, load balancers, firewalls, processors, modules, Customer Premises Equipment (CPE), or any other suitable device, component, element, or object operable to exchange information within communication system 100. In various embodiments, router 110 can be deployed in fixed or mobile environments. In a fixed environment, for example, the router 110 may be static, such as in a building, tower, stadium, and so forth. In a mobile environment, for example, the router 110 may be in motion or mobile, such as in a bus, airplane, train, ship, or the like.

In various embodiments, one or more I/O interfaces can be added to router 110 or removed from router 110 to provide one or more primary links for transferring data or information within network 130. In various embodiments, one or more I/O interfaces can be added to router 110 or removed from router 110 to provide one or more secondary (or "backup") links that can be used for the exchange of data or information within network 130 in the event that an exchange of data or information on one or more primary links is problematic. In various embodiments, such issues can include, but are not limited to: loss of connectivity to the main link, congestion on the main link, loss of equipment connected to the main link (e.g., equipment failure, power outage, etc.), interference on the main link, data errors, combinations thereof, and the like.

The first I/O124.1 interface can provide a first path for one or more cellular links to the eNodeB 132. In this description, to illustrate certain features of router 110, the first I/O124.1 interface can be alternatively referred to as "cellular interface" 124.1, "first cellular interface" 124.1, "cellular I/O interface" 124.1, or "first cellular I/O interface" 124.1. As referred to in this specification, the term "cellular" may refer to Long Term Evolution (LTE) communications, such as evolved Universal Mobile Telecommunications System (UMTS) terrestrial radio access network (E-UTRAN) communications, generally referred to as 4G, LTE and/or LTE-advanced (LTE-a) as specified by the 3 rd generation partnership project (3 GPP).

In some embodiments, the router 110 can be configured with multiple cellular I/O interfaces, such as a second I/O124.2, which can provide a second path for one or more additional cellular links to the eNodeB 132. In this specification, the second I/O124.2 can be alternatively referred to as a "cellular interface" 124.2, a "second cellular interface" 124.2, a "cellular I/O interface" 124.2, or a "second cellular I/O interface" 124.2. In general, an eNodeB (e.g., eNodeB 132) can be deployed to provide coverage for a 4G/LTE-a macro network, and/or a home (HeNB) (not shown) can be deployed to provide coverage for a 4G/LTE-a small cell access network. In general, small cell networks differ from macro networks in that: small cell networks are typically made up of multiple small cell access points (e.g., henbs) that can provide the closest coverage to users in environments (e.g., within buildings, structures, facilities, etc.) where macro network coverage is limited or interfered with.

As discussed in this specification, the cellular interface (e.g., I/O interfaces 124.1, 124.2) of router 110 may be recognized by a given eNodeB (e.g., eNodeB 132) or a given HeNB as a User Equipment (UE). For user equipment, it is meant that the eNodeB can recognize the cellular interface as a typical mobile subscriber or user that wants to initiate a flow in the communication system 100 over some network. For example, user equipment can include any component, element, or object capable of initiating voice, audio, video, media, or data exchanges within communication system 100. As used in this document, data and/or information refers to any type of numeric, voice, video, media, or script data, or any type of source or object code, or any other suitable information in any appropriate format that may be communicated from one point to another. In some embodiments, router 110 cellular interfaces (e.g., one or more I/O interfaces 124.1 and/or 124.2, depending on the configuration) may have a bundled subscription to network access and/or application services. In various embodiments, the cellular interface IP addresses of one or more I/O interfaces 124.1 and/or 124.2 can be specified using Dynamic Host Configuration Protocol (DHCP), stateless address auto-configuration, default bearer activation, etc., or any suitable variant thereof (as specified in 3GPP TS 29.061). In general, a given cellular I/O interface (e.g., any of the one or more I/O interfaces 124.1 and/or 124.2) can be associated with an entity, such as an International Mobile Subscriber Identity (IMSI), that can be associated with an operator or subscriber that operates the I/O interface of router 110.

The third I/O interface 124.3 can provide one or more wireless links to the WLAN AP. As referred to herein, the term "wireless" may refer to any wireless communication, such as Wi-Fi, WiMAX, Bluetooth^TM、

Combinations thereof, and the like. In general, a WLAN AP (e.g., WLAN AP 136) can be deployed to provide access to wireless technologies (e.g., Wi-Fi, WiMAX, Bluetooth^TM、

Etc.).

The third I/O interface 124.3 can provide one or more wired links to the network node 138. As referred to herein, the term "wired" may refer to any wired communication such as Ethernet, Power over Ethernet (PoE), Digital Subscriber Line (DSL), cable, fiber optics, fiber channel, Universal Serial Bus (USB), Serial, T1/E1/T3/E3/OCn (OC3, OC48, etc.), combinations thereof, and the like. In various embodiments, the wired network node 138 can be one or more of the following: a network appliance, a server, a switch, a gateway, a bridge, a load balancer, a firewall, a processor, a module, a Customer Premises Equipment (CPE), or any other suitable device, component, element, or object operable to exchange information within communication network 130.

It should be noted that the configuration of the multiple I/O interfaces 124.1-124. N and the various I/O interfaces supporting certain link types (e.g., cellular, wireless, wired) is provided for illustrative purposes only and is not intended to limit the scope of embodiments of the present disclosure. It is within the scope of the present disclosure that any number and/or combination of I/O interfaces can be configured for router 110, depending on operator needs.

Router 110 can include one or more elements or components to facilitate the operation of router 110. As shown in FIG. 1A, host router interconnect 112 is capable of providing interconnections between various elements, logic, etc. within router 110. In various embodiments, host router interconnect 112 can include one or more electronic buses, one or more electronic circuits, hardware, software, combinations thereof, or the like, and host router interconnect 112 can facilitate the transfer of data, information, or the like between various elements, logic, or the like within router 110. In various embodiments, host router logic 118 can comprise one or more electronic circuits, hardware, software, combinations thereof, and the like, as may be configured for router 110, host router logic 118 may facilitate various operations of router 110 and/or any combination of I/O interfaces 124.1-124. N. In certain embodiments, host router logic 118 may include router cellular interface logic that can be used by a router to set the cellular interface bandwidth of one or more cellular I/O interfaces. In various embodiments, host router storage 120 can include one or more databases, look-up tables, lists, combinations thereof, and the like, and host router storage 120 can be accessed (e.g., written to and/or read from) to facilitate one or more operations of router 110. In various embodiments, memory element 116 is capable of storing data and/or information that may be used to implement the dynamic bandwidth adjustment, I/O interface 124.1-124. N, and/or I/O interface operations of router 110 and/or other operations listed herein. Additionally, in certain embodiments, processor 114 can be a hardware processor capable of executing software, instructions, algorithms, combinations thereof, and the like, to perform the dynamic bandwidth adjustment listed herein, the I/O interface operations of router 110, and/or other operations.

Referring to fig. 1B, fig. 1B is a simplified block diagram illustrating an I/O (e.g., cellular) interface 124.1 of router 110 according to one embodiment of router 110. As shown in FIG. 1B, I/O interface 124.1 includes a cellular modem 150, an I/O interconnect 160, a processor 162, a memory element 164, I/O interface logic 166, and I/O interface storage 168. The cellular modem 150 includes a transmitter 152 and a receiver 154, wherein the transmitter 152 can facilitate transmission of uplink traffic toward the eNodeB 132 and the receiver 154 can facilitate transmission of downlink traffic from the eNodeB 132. In various embodiments, the I/O interface 124.1 through the cellular modem 150 can include one or more antennas (not shown) that can facilitate the transmission and/or reception of uplink and/or downlink traffic.

In various embodiments, the I/O interconnect 160 can include one or more electronic buses, one or more electronic circuits, hardware, software, combinations thereof or the like, and the I/O interconnect 160 can facilitate the transfer of data, information or the like between the I/O interface 124.1 and various elements, logic or the like within the router 110. In various embodiments, the I/O interface logic 166 can comprise, among other things, one or more electronic circuits, hardware, software, combinations thereof, and the like, the I/O interface logic 166 can facilitate various operations of the I/O interface 124.1 and the router 110. In various embodiments, I/O interface storage 168 can include one or more databases, look-up tables, lists, combinations thereof, and the like, and I/O interface storage 168 can be accessed (e.g., written to and/or read from) to facilitate one or more operations of I/O interface 124.1 and router 110. In various embodiments, memory element 164 is capable of storing data and/or information to be used to implement the dynamic bandwidth adjustment, I/O interface 124.1 and/or I/O interface operations of router 110 and/or other operations listed herein. Additionally, in certain embodiments, processor 114 can be a hardware processor capable of executing software, instructions, algorithms, combinations thereof, and the like, to perform the dynamic bandwidth adjustment, I/O interface 124.1, and/or I/O interface operations of router 110, and/or other operations listed herein.

Before explaining in detail some of the operational aspects of fig. 1A-1B, it is important to understand the common nature of the cellular interfaces of routers that are typically provided in commercial architectures. The following basis is sincerely provided for teaching purposes only and therefore should not be construed as limiting the broad teachings of the present disclosure in any way. As previously discussed, the router (e.g., ISR) can include a 4G/LTE cellular interface, which can be used as a primary/backup link on the router. Depending on the I/O interfaces (e.g., different combinations of wired, wireless, and/or cellular interfaces) configured or included for the router, the information related to the interface bandwidth can be configured for the router. As discussed herein, "interface bandwidth" means a bandwidth set for an I/O interface of a router over which data or information can be exchanged between the router and another network element. In various embodiments, the interface bandwidth set for a given I/O interface can be used by another router to determine quality of service (QoS) configuration, timing, buffer management, load balancing, etc. for one or more data flows associated with the I/O interface.

Currently, the bandwidth set for the router's cellular interface is based on the technology to which the cellular modem is attached. For example, if a cellular modem is attached to a 4G/LTE network (e.g., to an eNodeB or HeNB), the cellular interface bandwidth is fixed and set to the theoretical maximum of 50Mbps (megabits per second) uplink bandwidth and 100Mbps downlink bandwidth (often expressed using the term "50/100 Mbps"). Unlike wired (e.g., fixed bandwidth) interfaces, the cellular interface bandwidth can be dynamically changed based on, for example, signal conditions (e.g., signal-to-noise ratio), the number of users in a cell (e.g., the coverage area of an eNodeB and/or HeNB), etc.

Setting the interface bandwidth of the cellular interface of the router to the theoretical maximum may cause operational inefficiencies/problems for the router, since the air interface is a shared resource (e.g., shared with other user equipment in the cell). In practice, the maximum bandwidth that can be achieved for a cellular interface may be much less than the theoretical maximum. Since data throughput can vary widely based on changes in bandwidth, it is difficult for critical applications to guarantee bandwidth when using a cellular interface. For example, setting the interface bandwidth of the cellular interface to a theoretical maximum may pose a problem in configuring quality of service (QoS) (e.g., sometimes referred to as a Hierarchical Queuing Framework (HQF)), because the configuration of HQF depends on the interface bandwidth (e.g., the percentage of bandwidth set by the Command Line Interface (CLI) of the router). The incorrect QoS configuration may result in poor QoS, affecting the delay, reliability, and/or jitter of user traffic associated with the cellular interface. Therefore, for features like QoS to be used properly on the router, the interface bandwidth should only be as accurate as possible.

According to various embodiments described herein, the router 110 is configured to provide systems and methods for providing dynamic bandwidth adjustment to a cellular interface. In at least one embodiment, the methods described herein can be performed by one or more hardware processors of router 110. In general, the systems and methods provided by the router 110 may enable the 4G/LTE cellular interface (e.g., the first I/O interface 124.1, the second I/O interface 124.2, depending on the configuration) to dynamically adjust the cellular interface bandwidth set for the router 110 based on actual signal conditions of the operating environment in which the router 110 operates. Router 110 (e.g., including a processor, logic, etc., among other things) can then make informed decisions for various operations based on interface capabilities rather than theoretical assumptions.

In various embodiments, the systems and methods provided by router 110 may predict 4G/LTE uplink throughput based on instantaneous signal conditions and network availability. In operation, for example, a 4G/LTE uplink throughput can be calculated based on the number of Resource Blocks (RBs) allocated by a given eNodeB (e.g., eNodeB 132) for uplink data traffic of a given user equipment (e.g., cellular I/O interface 124.1). As before, the eNodeB may simply identify the cellular routing interface as a User Equipment (UE) that wants to initiate a flow through a certain network in the communication system 100. Thus, the terms "user equipment", "cellular interface", and "cellular I/O interface" can be used interchangeably in this specification.

According to the 3GPP standard, RBs can be used to transmit resources to one or more UEs in a cell. The resources can include control resources and data resources. An RB consists of several resource elements, each of which can be allocated within a symbol for a particular subcarrier (e.g., frequency) associated with a given UE. Typically, two RBs are included in each subframe of data transmitted in a particular output time interval (TTI), which typically lasts 1 millisecond (msec) in 4G/LTE communications. The RBs are generally referred to as gaps in a subframe, where a gap typically lasts 0.5msec (e.g., two gaps or RBs per subframe).

The minimum unit of an RB is a resource element, which represents one subcarrier by one symbol. Each RB (or gap) can include six (6) or seven (7) symbols depending on the cyclic prefix configuration. For 4G/LTE, the number of subcarriers of an RB is 12, with each extending 15 kilohertz (15 KHz bandwidth). Each resource element of each symbol of the RB can be represented using a number of bits, which can vary with a Modulation Coding Scheme (MCS) selected for communication. For example, 6 bits/symbol can be used for 64 Quadrature Amplitude Modulation (QAM) depending on the MCS selected for the communication; for 16 QAM, four (4) bits/symbol can be used; two (2) bits/symbol, etc. can be used for Quadrature Phase Shift Keying (QPSK).

In various embodiments, the predicted uplink throughput can be used to configure the cellular interface bandwidth and can be dynamically adjusted as and/or when the eNodeB allocation for the UE (e.g., I/O interface 124.1) changes. According to current 3GPP standards, e.g., as generally specified by Technical Specifications (TS)36.201, TS 36.211, TS 36.212, TS 36.213, TS 36.214, and TS 36.216, the process of scheduling uplink data traffic for a UE can include:

1) when data is to be transmitted from a given UE (e.g., the first I/O interface 124.1) to a given eNodeB (e.g., eNodeB 132), the UE can transmit a Scheduling Request (SR) to the eNodeB using a Physical Uplink Common Channel (PUCCH). A Buffer Status Report (BSR) indicates the amount of data that the UE needs to send to the eNodeB;

2) the eNodeB can estimate the uplink signal quality using several methods: a Sounding Reference Signal (SRS) method, a UE Demodulation Reference Signal (DRS) method, or a hybrid automatic repeat request (HARQ) feedback method;

a) for the SRS method: the UE can transmit (e.g., by transmitter 152) a predefined sequence across certain RBs across the entire channel bandwidth, and the radio resources of the eNodeB are used by the eNodeB to determine signal quality;

b) for the DRS method: the UE can transmit in the middle symbol of each gap (e.g., RB), the radio resources used by the eNodeB to determine signal quality;

c) for the HARQ feedback method: the UE can start transmitting RBs using a low Modulation Coding Scheme (MCS) until the block error rate (BLER) starts to increase to determine the signal quality. Generally, no radio resources are used to determine signal quality using this method;

3) based on the signal quality determined using one or more of the above methods, the eNodeB will indicate to the UE (e.g., the first I/O interface 124.1) the number of Resource Blocks (RBs) and the modulation coding scheme used for a particular uplink transmission by sending an Uplink (UL) GRANT (GRANT) message to the UE on a Physical Downlink Common Channel (PDCCH). The eNodeB can send multiple UL GRANT messages for multiple UL transmissions scheduled for the UE.

4) The UE (e.g., the first I/O interface 124.1) can transmit data on the RB using a coding scheme required by the eNodeB (e.g., based on signal conditions).

Because the UE 4G/LTE cellular modem (e.g., the cellular modem 150 of the I/O interface 124.1) knows the RBs allocated by the eNodeB and the modulation coding scheme required by the eNodeB, the Instantaneous Throughput (IT) of the I/O interface 124.1 can be calculated based on the determined throughput of one (1) resource block multiplied by the number of RBs to be used for UL transmission. In various embodiments, the throughput of 1 RB is determined by 1) multiplying the number of symbols to be used for the particular RB to be transmitted by 2) multiplying the number of bits per symbol to be used for each symbol (e.g., determined by the modulation and coding scheme) by 3) the number of subcarriers of that RB [ e.g., number of symbols bits/symbol ] the number of subcarriers transmitted in the time frame of the RB ].

After determining the throughput per RB, in various embodiments, the instantaneous throughput can be averaged based on the number of UL GRANT messages received by the cellular I/O interface 124.1 to determine a predicted average throughput for the cellular I/O interface 124.1. In some embodiments, the predicted average throughput can be used by router 110 to dynamically change, update, and/or set the cellular interface bandwidth of router 110 based on the actual uplink signal quality. In some embodiments, the average throughput by which the cellular interface bandwidth can be dynamically changed, updated, and/or set for router 110 via the router cellular interface logic can be communicated to processor 114, host router logic 118, and the like.

Consider an operational embodiment assuming that each RB lasts 0.5msec (in the time domain) and contains 12 subcarriers, each having a bandwidth of 15 kilohertz (kHz) (in the frequency domain). As noted, the data throughput per 1 RB (TperRB) can be determined by multiplying the number of symbols by the modulation coding scheme used per subcarrier per RB. As for the best case, for example, it is considered that 7 symbols using a 64 QAM MCS (e.g., 6 bits/symbol) can be transmitted within 0.5 msec. Using the above equation, this translates to a throughput per 1 RB (tperb) of approximately 1 megabit per second per RB (Mbps/RB) [ e.g., 7 symbols/RB 6 bits/symbol 12 subcarriers transmitted per RB time frame (0.5msec) ]. The Instantaneous Throughput (IT) can be calculated by multiplying the number of RBs transmitted for UL GRANT by 1MBps/RB TperRB [ e.g., number of RBs TperRB ] through the cellular I/O interface 124.1. Say, 6 RBs per UL GRANT approximately 1Mbps/RB translates to approximately 6 Mbps. The predicted average throughput can be determined by determining the total number of instantaneous throughputs divided by the number of UL GRANT messages received by I/O interface 124.1 [ e.g., (IT1+ IT2+. + ITm)/(the number of received UL GRANT messages), where "m" is an integer equal to the number of received UL GRANT messages ]. The predicted average throughput calculated by I/O interface 124.1 (e.g., by processor 162, I/O interface logic 166, modem firmware, etc.) can be sent to processor 114 and/or host router logic 118, and the cellular interface bandwidth configured for first I/O interface 124.1 can be dynamically changed, updated, and/or set. In various embodiments, the router 110 can include cellular interface logic (e.g., within or separate from the host router logic) that can be used to dynamically change, update, and/or set the cellular interface bandwidth configured for the first I/O interface 124.1.

As noted above, since signal quality can vary depending on various factors, the MCS of each UL GRANT can vary, and accordingly the instantaneous throughput of each UL GRANT can vary. Thus, the predictive average throughput calculation can be used to provide a "snapshot" of the signal conditions over a period of time. In various embodiments, the period of time within which the predicted average throughput may be calculated and/or used to change, update and/or set the cellular interface bandwidth for a given cellular I/O interface can vary depending on the environment (e.g., fixed or mobile), router 110 loading, environmental conditions, operating conditions, combinations thereof, and so forth.

For example, in a fixed environment with generally predictable environmental and/or operational conditions, the update rate of the time frame used to calculate the predicted average throughput for a particular cellular interface and update the bandwidth of the cellular interface can be approximately 1 minute or more, depending on the estimated environmental and/or operational conditions (e.g., with more users in an open-time environment). In another example, say, in a mobile environment, the update rate may be reduced to a range from approximately 20-30 seconds up to 1 minute or more, for example, based on estimated environmental and/or operating conditions (e.g., peak hours of commuting for a mass transit system). Too frequent updates to the cellular system in determining the appropriate time frame may be seen as a dynamic raising of the cellular interface within a short time frame that may reduce the performance and/or efficiency of the cellular interface and/or router.

Thus, in one or more embodiments, the system and method provided by router 110 can provide various advantages, including but not limited to: provide higher layer features of router 110, better visibility of the characteristics of the cellular I/O interface; adjusting one or more features provided by router 110; ensuring that critical applications of the router and/or cellular interface can have bandwidth guarantees under rapidly changing conditions; providing adaptive QoS features for the cellular I/O interface (e.g., providing percentage-based traffic monitoring of interface bandwidth for different traffic types, which can be adjusted accordingly based on the cellular interface bandwidth set for the cellular I/O interface without any user reconfiguration of the router); providing load balancing among the plurality of cellular I/O interfaces based on a predicted average throughput of the cellular I/O interfaces; and/or to provide a method of reducing the delay of packets during cellular modem congestion of a cellular I/O interface.

Turning to fig. 2, fig. 2 is a simplified flowchart illustrating example operations 200 associated with providing dynamic bandwidth adjustment to a cellular interface in accordance with one possible embodiment of the audio recording 110. In various embodiments, operations 200 can be performed by any combination of processor(s) and/or logic of router 110 (e.g., processor 114 and/or host router logic 118, in some embodiments host router logic 118 can include router cellular interface logic) and/or cellular I/O interfaces (e.g., I/O interface logic 166 and/or processor 162 of cellular I/O interface 124.1).

The predicted average throughput can be calculated for one or more I/O interfaces (e.g., cellular I/O interfaces 124.1, 124.2) whenever configured (e.g., according to deployment environment, update rate, etc.). Thus, operations may begin at 202, where a predicted average throughput is determined for each of one or more cellular interfaces of a router (e.g., router 110). At 204, the operations can include adjusting a bandwidth of a cellular interface of the router based at least in part on the determined predicted average throughput for each of the one or more cellular interfaces, and the operations can end. In various embodiments, the determining and adjusting steps can be repeated according to an update rate, which can be configured by a network operator, a device manufacturer, a user, a combination thereof, and so forth. In some embodiments, the update rate can depend at least in part on whether the router is deployed in a mobile environment or a fixed environment.

Turning to fig. 3, fig. 3 is a simplified flow diagram illustrating example operations 300 associated with determining a predicted average throughput for a cellular interface (e.g., cellular I/O interface 124.1) of router 110. In various embodiments, operations 300 can be performed by any combination of hardware processor(s) and/or logic of router 110 (e.g., processor 114 and/or host router logic 118, in some embodiments host router logic 118 can include router cellular interface logic) and/or cellular I/O interfaces (e.g., I/O interface logic 166 and/or processor 162 of cellular I/O interface 124.1). The predicted average throughput can be calculated for a particular cellular I/O interface (e.g., cellular I/O interface 124.1) whenever configured (e.g., according to deployment environment, update rate, etc.).

Accordingly, operations may begin at 302, where one or more per resource block throughput values may be determined for a cellular interface. In various embodiments, each of the one or more per resource block throughput values can be associated with a particular resource block associated with a particular uplink grant received by the cellular interface. In various embodiments, determining the per resource block throughput value for a particular resource block may be based on the number of symbols to be used to transmit the particular resource block, the number of bits per symbol for the resource block, the number of subcarriers included in the resource block per RB time frame. In particular embodiments, the number of bits per symbol is based on the modulation coding scheme of a particular resource block.

At 304, the operations can include determining one or more instantaneous throughput values of the cellular interface based on each of the one or more per resource block throughput values and a number of resource blocks to transmit for each of the one or more uplink grants received by the cellular interface. At 306, the operations can include determining a predicted average throughput for the cellular interface based on a total of one or more instantaneous throughput values related to a number of the one or more uplink grants received by the cellular interface, and the operations can end.

Turning to fig. 4, fig. 4 is a simplified flowchart illustrating example operations 400 associated with providing adaptive quality of service for a cellular interface in accordance with one possible embodiment of router 110. Before explaining the example operation of fig. 4 in detail, it is important to understand the common nature of the current quality of service (QoS) models of the I/O interfaces of routers. The following information is sincerely provided for instructional purposes only, and thus should not be construed in any way as limiting the broad teachings of the present disclosure.

With the current QoS model, a single service policy is applied to the I/O interface, which is static. For a wired interface this approach is generally acceptable, but for a cellular interface, as discussed herein, bandwidth can vary based on the number of users, signal strength, signal-to-noise ratio, whether the router is in a stationary environment, whether it is a mobile environment, and so on. Moreover, having a static QoS policy associated with the cellular interface does not provide control of QoS to account for changes based on the current availability of bandwidth of the cellular interface.

According to various embodiments described herein, router 110 can be configured to create multiple QoS policy mappings to predefined levels based on an expected availability of bandwidth for cellular I/O interfaces (e.g., cellular I/O interfaces 124.1, 124.2, etc., depending on the configuration) and one or more QoS service policies configured for router 110 (e.g., gold service policies, silver service policies, bronze service policies, etc.).

Consider, for example, that the QoS service policies configured for different levels of traffic are as follows:

gold-80% of the Bandwidth (BW) of a particular cellular interface is allocated to high priority voice traffic, 20% of which is allocated to critical data, and a "best effort" is applied to all default levels of traffic;

silver-50% of the BW for a particular cellular interface is allocated to high priority voice traffic, 40% of which is allocated to critical data, and 20% of which is allocated to a default level;

bronze — 30% of the BW for a particular cellular interface is allocated to high priority voice traffic, 20% of which is allocated to critical data, 10% of which is allocated to bulk data, and the remaining BW is allocated to a default level.

In various embodiments, a QoS policy map can be configured based on an expected availability of bandwidth of a cellular interface of router 110 (e.g., based on an expected fixed or mobile deployment, an expected environmental condition, an expected operational condition, etc., expectations, etc.), and can be used to associate a given QoS service policy to the cellular interface during operation when the cellular interface bandwidth is set based on a predetermined average throughput of the cellular interface. In various embodiments, various cellular interface bandwidth thresholds can be configured for the QoS policy mapping, by which a comparison with a cellular interface bandwidth set for a cellular I/O interface (e.g., adjusted/set according to a predicted average throughput) can be performed to determine QoS service policies to associate to the cellular I/O interface for one or more flows processed by the interface. In various embodiments, the QoS policy map(s), QoS service policies, and/or associated level maps can be configured by a network operator, device manufacturer, user, combinations thereof, and so forth, depending on the deployment of router 110.

Consider an operational example involving the example QoS service policies discussed above. During operation, the foregoing gold service policy can be associated to the cellular interface when the cellular interface bandwidth is low (e.g., adjusted below a predetermined "low" BW threshold). When BW is limited, gold service policies can be used to ensure that 80% of the cellular interface BW can be allocated to high priority traffic. Continuing with the present example, the above-described silver service policy can be associated to the cellular interface when the interface bandwidth increases (e.g., is adjusted to be above the predetermined "low" BW threshold but below the predetermined "high" BW threshold). The above-described bronze service policy can be associated to the cellular interface when the interface bandwidth is further increased (e.g., adjusted above a predetermined "high" BW threshold). In various embodiments, dynamically allocating BW to different levels of traffic depending on the bandwidth set for an interface can help ensure that critical applications can be provided with sufficient bandwidth on a given cellular interface.

Returning now to fig. 4, fig. 4 illustrates operations 400 associated with providing adaptive quality of service for a cellular I/O interface in accordance with one possible embodiment of router 110. With respect to fig. 4, it is assumed that at least one QoS policy map comprising a plurality of QoS service policies, wherein each QoS service policy comprises a respective level map, has been configured for a given cellular interface (e.g., first I/O interface 124.1) of router 110. In various embodiments, the QoS policy mappings, QoS service policies, and/or level mappings can be configured and/or stored in memory element 116, host router storage 120, memory element 164, I/O interface storage 168, or a combination thereof. In various embodiments, the processor 114, the host router logic 118, the processor 162, the I/O interface logic 166, or a combination thereof can be used.

Whenever, depending on the configuration (e.g., depending on the deployment environment, update rate, etc.), a predicted average throughput can be calculated for a particular cellular I/O interface (e.g., cellular I/O interface 124.1). At 402, operations can include determining a bandwidth to adjust for a cellular interface. At 404, the operations can include comparing the bandwidth adjusted for the cellular interface to at least one QoS policy map configured for the cellular interface. In at least one embodiment, the QoS policy map includes a plurality of QoS service policies, each QoS service policy associated with a particular bandwidth threshold. In at least one embodiment, each QoS service policy also includes an allocation of cellular interface bandwidth to a plurality of traffic levels or flows that may be handled by the cellular interface. At 406, the operations include associating a particular QoS service policy to the cellular interface based on the comparison result, and the operations may end. In various embodiments, operation 400 can be repeated after determining that the bandwidth of one or more cellular interfaces has been adjusted.

In various embodiments, providing adaptive QoS over a cellular interface can provide one or more advantages, including but not limited to: associating the level of policy in a dynamic manner can help make router operation more reliable than a static configuration (which is particularly important when the router is deployed in a moving environment where the cellular interface may be the only primary interface of the router and/or under rapidly changing conditions), providing adaptive QoS over the cellular interface can help ensure that critical applications have sufficient bandwidth.

Turning to fig. 5, fig. 5 is a simplified block diagram illustrating details associated with dynamic load balancing of cellular I/O interfaces in accordance with one possible embodiment of router 110. As shown in FIG. 5, router 110 can include host router interconnect 112, processor 114, memory element 116, host router logic 118, host router storage 120, I/O interfaces 124.1-124. N, and load balancing logic 210. As shown in fig. 5, the network 130 includes an eNodeB 132, a WLAN AP136, and a wired network node 138.

Before detailing some of the operational aspects of fig. 5, it is important to understand the common nature of load balancing of cellular interfaces of routers that is typically provided in commercial architectures. The following basis is sincerely provided for teaching purposes only and therefore should not be construed as limiting the broad teachings of the present disclosure in any way. As discussed herein, the data throughput over the cellular interface can vary based on signal quality, the number of users within a cell, the cell bandwidth, and so forth. On the router, if there are multiple 4G/LTE interfaces, the router Internal Operating System (IOS), logic, etc. will identify these cellular interfaces as equivalent (e.g., having the same path metrics as it relates to making routing decisions) based on their data throughput capabilities. In practice, however, multiple 4G/LTE interfaces connected to the same cell tower (e.g., eNodeB) can behave differently in terms of uplink data throughput capability. For example, the uplink data throughput capability can depend on the amount of resources (e.g., resource blocks) that the eNodeB allocates to each 4G/LTE modem.

Thus, in current deployments, for per-packet load balancing between a pair of 4G/LTE cellular interfaces, the router only assumes that each 4G/LTE interface will bear 50% of the load, when in fact one 4G/LTE interface may be able to handle higher data throughput than the other 4G/LTE interface, and vice versa. Such assumptions can make load balancing of routers inefficient.

Several protocols, including Interior Gateway Routing Protocol (IGRP) and enhanced IGRP (eigrp), support non-peer load balancing between interfaces using a "variance" command, which provides a static configuration that is used to route packets between multiple paths to the same destination. For example, if there are two paths to a destination, EIGRP computes path metrics based on the bandwidth and delay of each interface associated with each path, and the variance can be configured to provide unequal load balancing between the paths.

Consider a numerical example. For example, the first path has a calculated first path metric of size 20 (referred to herein as "P1"), and the second path has a calculated second path metric of size 50 (referred to herein as "P2"). In the current example, because P1 may have a lower metric to the same destination, P1 will be entered into the router's routing table and all traffic to that destination will use P1. If unequal load balancing is provided between P1 and P2, the minimum metric used to place the route into the routing table should be made greater than the P2 path metric (e.g., 50). In this example, a variance of size 3 would need to be added to the load balancing configuration, which would result in a path metric of 3 x 20-60 through P1. Now P1 and P2 can be used for load balancing packet boundaries for the destination and can perform unequal load balancing based on the metrics provided. In the current example, for each packet sent through P2, 3 packets will be sent through P1.

However, as noted, in current deployments, non-peer-to-peer load balancing between cellular interfaces is not supported. Since the bandwidth of the cellular interfaces is assumed to be the theoretical maximum, the load is only shared equally across the various cellular interfaces, which may be inefficient for load balancing the routers.

According to various embodiments described herein, router 110 may provide systems and methods that facilitate dynamic load balancing between cellular interfaces. In essence, when multiple cellular interfaces (e.g., the first I/O interface 124.1 and the second I/O interface 124.2) are configured for the router 110, the router 110 utilizes knowledge of the cellular interface bandwidth to make intelligent per-packet load balancing decisions. For example, the router 110 can leverage the data throughput capabilities of the 4G modems (e.g., for the uplink) of multiple cellular interfaces and make intelligent load balancing decisions.

In various embodiments, the variance of one or more cellular interfaces can dynamically vary based on the interface bandwidth determined for each particular interface (e.g., first I/O interface 124.1 and second I/O interface 124.2), which may cause optimal multipath to be used for non-peer load balancing of routers 110.

During operation, the interface bandwidth for each cellular interface can be calculated based on the data throughput (uplink) capability of each respective 4G modem. For a plurality of cellular interfaces, the variance of the path metrics between the plurality of cellular interfaces can be calculated as the ratio between the minimum path metrics of the cellular interfaces and the maximum path metrics of the cellular interfaces. In various embodiments, the ratio can be calculated as an integer and oriented up to the next integer as shown in equation 1 below.

Variance + (max (P1, P2.. Pm)/min (P1, P2.. Pm)) +

(value rounded to the next integer) equation 1

For equation 1, P1 is the path metric of the destination (e.g., eNodeB 132) using the first cellular interface (e.g., cellular interface 124.1), P2 is the path metric of the second cellular interface (e.g., cellular interface 124.2), and so on up to the "m" th cellular interface. Router 110 can include a routing table in which the variance of the path metric can be updated based on changes in the calculated variance.

In various embodiments, the path metric for each path can be calculated as a function of bandwidth and delay for each interface, according to either the EIGRP and/or IGRP protocols. In general, the path metric for a path can be determined by adding the bandwidth and delay of the path (which can be scaled to approximately produce a 32-bit metric) and multiplying by the load of the path. In some embodiments, router 110 can be configured with an algorithm that can be used to determine whether the variance of the path metric between cellular interfaces changes, since the variance can be calculated at various intervals. If the variance of the path metric changes, the new variance can be used to update the routing tables of the router 110 for multiple cellular paths. In various embodiments, a variance update can be provided in a manner similar to that described above, wherein the variance can be applied to the minimum path metric value in the routing table to provide the minimum path metric value between cellular interfaces. Table 1, shown below, illustrates an example algorithm that can be used to determine variance changes between multiple cellular interfaces of router 110.

TABLE 1

It is noted that the 10 second sleep interval shown in table 1 is provided for illustrative purposes only and is not intended to limit the broad scope of the teachings of the present disclosure. It should be appreciated that the interval can vary based on deployment, environmental conditions, operating conditions, combinations thereof, and the like. In some embodiments, the variance calculation may be triggered in response to receiving that the bandwidth of one or more cellular interfaces has been adjusted.

Thus, in various embodiments, router 110 can provide for dynamically varying variances for particular destinations, which can cause routes to be included within early routing tables, and router 110 can perform non-peer load balancing. In one or more embodiments, the solutions described in the present disclosure may provide one or more advantages, including but not limited to: the actual throughput/bandwidth of the router-based cellular interface provides different routing metrics, allowing the router to achieve better load balancing between cellular interfaces and to reduce packet delay and/or jitter through the network.

Turning to fig. 6, fig. 6 is a simplified flowchart illustrating example operations 600 associated with providing dynamic load balancing of a cellular interface in accordance with one possible embodiment of router 110. In various embodiments, operation 600 can be performed using processor 114, host router logic 118, load balancing logic 510, processor 162, I/O interface logic 166, combinations thereof, and so forth, depending on the number of cellular I/O interfaces configured for router 110 (e.g., each I/O interface can include its own processor, logic, and so forth).

At 602, the operations can include calculating an updated variance of the path metric based on the path metric determined for each of the plurality of cellular interfaces of the router 110. In various embodiments, the path metric determined for each of the plurality of cellular interfaces can be based at least in part on the adjusted bandwidth and associated with each interface. At 604, the operations can include comparing the updated variance to a previous variance stored for the cellular interface (e.g., a previously calculated updated variance). At 606, the operations can include determining whether there is a difference between the updated variance and a previous variance stored for the cellular interface. If there is a discrepancy, then operations at 608 can include updating the routing table of the cellular interface using the updated variance. In various embodiments, updating can include applying the updated variance to a minimum path metric value associated with a particular cellular interface included in the routing table. At 610, the updated variance is stored (e.g., for use in subsequent comparisons). At 612, operating can include waiting for expiration of a predetermined time interval or receiving an indication of an adjusted bandwidth of a particular cellular interface before returning to 602 to determine another updated variance.

If operation determines at 606 that the updated variance is equal to the previous variance, operation continues to 612 to await expiration of a predetermined time interval or to receive an indication of adjusted bandwidth, upon which operation returns to 602.

Turning to fig. 7A-7B, fig. 7A-7B are simplified block diagrams illustrating additional details associated with one possible embodiment of a cellular modem of a router (e.g., the cellular modem 150 of the first I/O interface 124.1). As shown in fig. 7A, the cellular modem 150 can include a transmitter 152 and a receiver 154. In certain embodiments, transmitter 152 may receive uplink traffic (e.g., IP packets) to be transmitted to eNodeB 132 via host router interconnect 112, receiver 154 may receive downlink traffic from eNodeB 132 and may route the traffic to host router interconnect 112, only further via a router to another interface process, and so on. Each of the transmitter 152 and the receiver 154 can include a protocol stack that includes various layers and can enable IP packets to be transmitted or received via the cellular modem 150. For purposes of discussing certain features, fig. 7A only shows the layers associated with the sender 152, which can be configured for the router 110 to provide reduced delay of uplink packets during times when the cellular modem 150 is congested.

Thus, as shown in fig. 7A, the transmitter 152 can include a Packet Data Convergence Protocol (PDCP) layer 710, a Radio Link Control (RLC) layer 712, a Medium Access Control (MAC) layer 714, and a Physical (PHY) layer 716. Each layer may operate on IP packets to be sent. Generally for transmission of IP packets, the PDCP layer 710 can provide IP packet compression/decompression, delivery of user data, and the like; the RLC layer 712 can provide error correction, transfer of Packet Data Units (PDUs), segmentation, etc.; the MAC layer 714 can provide packet scheduling; and PHY layer 716 can provide a wireless interface to transmit packets to eNodeB 132.

MAC layer 714 can additionally include MAC control logic 720 and buffers 722, which can provide buffered packets to be scheduled for transmission via PHY layer 716. Fig. 7B shows example details associated with buffer 722. As shown in fig. 7B, buffer 722 can provide buffering of uplink traffic (e.g., IP packets 730) scheduled for transmission. A low water mark (LW) threshold 740 and a high water mark (HW) threshold 750 can be dynamically set for the buffer 722. In various embodiments, a low water mark (LW) threshold 740 and a high water mark (HW) threshold 750 may be associated with respective byte-level memory locations (e.g., physical or virtual memory addresses) determined for the buffer 722, the low water mark (LW) threshold 740 and the high water mark (HW) threshold 750 being capable of being used to indicate an amount of uplink traffic contained in the buffer. In various embodiments, low water mark (LW) threshold 740 and high water mark (HW) threshold 750 may each be dynamically adjusted based on the uplink bandwidth set for cellular interface 124.1 by one or more bandwidth adjustments.

Before detailing the additional operational aspects of fig. 7A-7B, it is important to understand the common nature of the MAC buffers of the cellular interface that are typically provided in commercial architectures. The following basis is sincerely provided for teaching purposes only and therefore should not be construed as limiting the broad teachings of the present disclosure in any way. Generally, cellular modems are designed for the smart phone market, which only considers a single user. However, when used on a router, it is important to have better control over the traffic flowing through the cellular modem. When data packets are transmitted to the cellular modem, the modem will buffer the packets in the transmit loop (e.g., buffer) of the MAC layer and send scheduling requests to the eNodeB via the PHY layer to schedule the packets on the uplink radio bearer.

Currently, there is no backpressure mechanism in the 3GPP standard to throttle the user interface (throttle) in case of delay in scheduling uplink traffic and accumulation (build) increase in internal buffers in the MAC layer. Because head-of-line ("head-of-line") blocking of lower priority packets (e.g., lower priority packets are stuck in a buffer blocking higher priority packets from being transmitted), the increase in buffer accumulation can cause a delay in high priority traffic to be transmitted over the cellular interface. The problem is further compounded when the radio link characteristics can fluctuate over time.

According to various embodiments described herein, the router 110 is configured to provide systems and methods to reduce the delay of packets during congestion in a cellular modem (e.g., the cellular modem 150 of the first cellular I/O interface 124.1). In at least one embodiment, the methods described herein are capable of being performed by one or more hardware processors of router 110. In general, the systems and methods provided by router 110 may allow the cumulative size in internal buffers in the UE's MAC layer (e.g., buffers 722 in MAC layer 714 of cellular I/O interface 124.1) to be dynamically changed based on the interface bandwidth, thereby reducing the latency of high priority traffic over the cellular interface.

In various embodiments, the size of the buffer 722 accumulation in the MAC layer 714 can be dynamically adjusted by adjusting the low water mark threshold 740 and the high water mark threshold 750. In some embodiments, a high water mark threshold 750 can be used to indicate the maximum number of packets that can be queued in buffer 722. In some embodiments, the low water mark threshold 740 can be used to indicate to a host router (e.g., router 110, including the processor/logic contained therein, the cellular interface is configured for router 110) that packets can now be enqueued to the internal buffer. In some embodiments, the high/low water flag threshold can be used to account for the hysteresis (hystersis) of the system.

In various embodiments, MAC control logic 720 may monitor a water mark threshold related to the accumulation of IP packets 730 in buffer 722 and may provide congestion feedback to a host router (e.g., router 110, including the processor/logic contained therein, the cellular interface is configured for router 110). During operation, for example, when high water mark threshold 750 of buffer 722 is reached (e.g., because a packet being enqueued in the MAC layer is not being invoked in the uplink), MAC control logic 720 can interrupt the host router using a decongestion interrupt to indicate that congestion exists. In turn, the host router logic 118 can stop enqueuing packets into the cellular interface 124.1 until the low water mark threshold is reached (e.g., packets in the buffer are being sent and the buffer has been emptied to the low water mark threshold). Upon reaching the low water mark threshold 740, the MAC control logic 720 can use the decongestion interrupt to interrupt the host router at this point and the uplink traffic flow to the cellular I/O interface 124.1 can resume. In various embodiments, the decongestion interrupt can be a control word, bit, byte, etc. that indicates that the high water mark threshold or the low water mark threshold has been reached.

In various embodiments, the levels of the low water mark (LW) threshold 740 and the high water mark (HW) threshold 750 can be dynamically changed based on the uplink bandwidth of the cellular I/O interface 124.1. For example, when the uplink bandwidth is high, the high/low water mark threshold can be made higher. Conversely, when the uplink BW is low, the high/low water mark threshold can be made lower. Such adjustments will help ensure substantial consistency and low latency for high priority traffic when Low Latency Queues (LLQs) are configured on the cellular interface of the router.

In various embodiments, a counter can be used to indicate how many times the high water mark/low water mark threshold is reached. In current deployments, if the cellular interface is unable to process the packet (e.g., due to resource exhaustion), the interface will silently drop the packet without any indication to the host. However, the system and method provided by router 110 can help ensure that packet loss does not occur in the UE, which can enable the host to provide a better count of packets.

In some embodiments, an algorithm can be used to calculate the low water mark threshold 740 and the high water mark threshold 750. In at least one embodiment, the high water mark threshold (measured in bytes) can be calculated as the product of the uplink Bandwidth (BW) (measured in bytes per second (Bps)) and the ideal delay for enqueued packets, as shown in equation 2 below.

High water mark threshold (measured in bytes) uplink _ BW (measured in bytes) ×

Ideal _ delay equation 2

For example, if the uplink bandwidth is equal to 1MBps and the ideal delay is 1 millisecond (msec), then the high water mark threshold is 1 x 10 x 6 x 1 x 10 x 3 is 1000 bytes. In some embodiments, the low water mark threshold can be calculated based on the delay of the system when eliminating congestion interruptions (e.g., stopping enqueuing packets in the cellular I/O interface 124.1). Thus, based on the change in the uplink bandwidth of the cellular I/O interface 124.1, the water mark threshold can be changed to provide a uniform delay of the changed uplink bandwidth.

In certain embodiments, the systems and methods provided by router 110 can help ensure relatively consistent and low latency to high priority traffic when a Low Latency Queue (LLQ) is configured for the router's cellular interface. Features like low delay queues (LLQ) typically rely on very fine control over the size of internal buffers to reduce the delay of packets passing through a router. Thus, the systems and methods provided by router 110 may facilitate tighter integration of the cellular interface internal MAC buffer with various router operations, thereby facilitating a better user experience during periods of congestion.

Turning to fig. 8, fig. 8 is a simplified flowchart illustrating example operations 800 associated with reducing packet delay during congestion in a cellular modem in accordance with one possible embodiment of router 110. In various embodiments, operation 800 can be performed using processor 114, host router logic 118, processor 162, I/O interface logic 166, MAC control logic 720, combinations thereof, and the like, depending on the number of cellular I/O interfaces configured for router 110 (e.g., each I/O interface can include its own processor, logic, and the like).

At 802, the operations can include monitoring a water mark threshold of a MAC buffer (e.g., buffer 722) of a cellular modem (e.g., cellular modem 150) of a cellular interface (e.g., first I/O interface 124.1). In various embodiments, the water mark threshold can include a high water mark threshold/a low water mark threshold set for the MAC buffer. In various embodiments, the high water mark threshold and/or the low water mark threshold can be set based on a number of bytes associated with the buffer.

At 804, the operations can include asserting an interrupt when a particular water mark threshold of the MAC buffer is reached. At 806, the operations can include adjusting enqueuing of the uplink packet into the MAC buffer based on the interruption. Operation may return to 802 to continue monitoring the water mark threshold. In various embodiments, an interrupt may be generated when the high water mark threshold is reached, which may provide an indication to stop the flow of uplink packets being sent to the cellular interface. In various embodiments, an interrupt can be generated when the low water mark threshold is reached, which interrupt can provide an indication to resume the flow of uplink packets being directed to the cellular interface after a previous indication to stop the flow of traffic being directed to the cellular interface.

Thus, as shown in FIGS. 1A-1B, 5, and 7A-7B, appropriate software and/or hardware is provisioned in the router 110 and/or I/O interfaces 124.1-124. N in order to facilitate the dynamic bandwidth adjustment for the cellular interface listed herein, as well as one or more other activities associated with the cellular interface. It is noted that in some embodiments, certain storage devices can be combined with memory elements (or vice versa), or storage devices can overlap/exist in any other suitable manner.

In one example implementation, router 110 is a network element intended to encompass a network appliance, server, switch, gateway, bridge, load balancer, firewall, processor, module, or any other suitable device, component, element, or object operable to exchange information that facilitates or otherwise facilitates coordinating and/or managing dynamic bandwidth adjustment and/or other associated activities of a cellular interface (e.g., such as those networks shown in fig. 1A and 5) as set forth herein. In other embodiments, these operations and/or features may be provided external to these elements, or included in some other network device, to achieve the intended functionality. Alternatively, one or more of these elements can include software (or reciprocating software) that can cooperate to achieve the operations and/or features listed herein. In other embodiments, one or more of these devices may include any suitable algorithms, hardware, software, components, modules, interfaces, or objects that facilitate the operations thereof. This may include appropriate algorithms and communication protocols that allow for the efficient exchange of data or information.

With respect to the internal structure associated with communication system 100, router 110 can include a memory element (e.g., memory element 116), and/or each of I/O interfaces 124.1-124. N can include a memory element (e.g., memory element 164 of first I/O interface 124.1) for storing data to be used to implement the dynamic bandwidth adjustment and/or other cellular interface operations listed herein (e.g., adaptive QoS, load balancing, congestion delay reduction, etc.). Additionally, router 110 can include a hardware processor (e.g., processor 114), and/or each of I/O interfaces 124.1-124. N can include a hardware processor (e.g., processor 162 of first I/O interface 124.1) that can execute software, instructions, algorithms, combinations thereof, and so forth to perform the dynamic bandwidth adjustment and/or other cellular interface activities discussed in this specification. These devices may also store information in any suitable memory element (e.g., Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), Application Specific Integrated Circuit (ASIC), etc.), software, hardware, or in any other suitable component, device, element, or object where appropriate and based on particular needs. Any memory items discussed herein should be read as being encompassed within the broad term "memory element". Information tracked or sent to the router 110 and/or any of the I/O interfaces 124.1-124. N can be provided in a database, register, control list, cache, or storage structure (all of which can be referenced in any suitable timeframe) (e.g., the memory element 116 and/or host router storage 120 of the router 110, the memory element 164 and/or I/O interface storage 168 of the first I/O interface 124.1, etc.). Any such storage options may be included within the broad term "memory element" as used herein. Similarly, any possible processing elements, modules, hardware and machines described herein should be construed as being encompassed within the broad term "processor". Each of the network elements and user devices can also include suitable interfaces for receiving, transmitting, and/or otherwise communicating data or information in a network environment.

It is noted that in certain example implementations, the dynamic bandwidth adjustment and/or other cellular interface operations listed herein may be implemented by logic encoded in one or more tangible media (e.g., host router logic 118 and/or load balancing logic 510 of router 110, I/O interface logic 166 and/or MAC control logic 720 of first I/O interface 124.1, etc.), which may include non-transitory media (e.g., in an ASIC, in Digital Signal Processor (DSP) instructions, software (possibly including object code and source code), and/or firmware to be executed by a processor, module, or other similar machine, etc.). In some of these examples, a memory element (e.g., memory element 116 of router 110, memory element 164 of first I/O interface 124.1, etc.) is capable of storing data for the operations described herein. This includes memory elements capable of storing software, firmware, logic, code, or processor instructions to be executed to implement the activities described herein. Processors, including hardware processors, can execute any type of instructions associated with data to implement the operations detailed herein. In one example, a processor (e.g., the

processors

114, 162 shown in fig. 1A-1B and 5) can transform an element or object (e.g., data) from one state or thing to another state or thing. In another embodiment, the activities set forth herein can be implemented using fixed logic or programmable logic (e.g., software/computer instructions executed by a processor, firmware, and/or controller), and the elements identified herein can be some type of programmable memory, programmable digital logic (e.g., a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an EPROM, an electrically erasable programmable read-only memory (EEPROM)), or an ASIC that includes digital logic, software, code, electronic instructions, or any suitable combination thereof.

It is noted that, in this specification, references to various features (e.g., elements, structures, modules, components, steps, operations, characteristics, etc.) included in "one embodiment", "an example embodiment", "an embodiment", "another embodiment", "some embodiments", etc., are intended to mean that any such features are included in one or more embodiments of the present disclosure, but may or may not be combined in the same embodiment. It is further noted that modules used in this specification can include executable files (which include instructions that can be understood and processed on a computer), and may also include library modules, object files, system files, hardware logic, software logic, or any other executable modules that are loaded during execution.

It is also important to note that the operations and steps described with reference to the preceding figures are only illustrative of some of the possible scenarios that may be performed by or within a system. Some of these operations may be deleted or removed where appropriate, or these steps may be modified or changed considerably without departing from the scope of the concepts discussed. In addition, the timing of these operations may be modified considerably, while still achieving the results taught by the present disclosure. The foregoing operational flows have been provided for purposes of illustration and discussion. The present system provides a great deal of flexibility in that any suitable scheduling, chronology, configuration, and timing mechanisms may be provided without departing from the teachings of the concepts discussed.

It is noted in the context of the examples provided above, as well as numerous other examples provided herein, that interaction may be described from the perspective of one, two, three, or four network elements. However, this is done for clarity and example purposes only. In some cases, one or more of these functions may be more easily described by close reference to network elements of limited data. It should be appreciated that communication system 100 (and its teachings) are readily scalable and can accommodate a large number of components, as well as more complex/sophisticated arrangements and configurations. Thus, the examples provided should not limit the scope or inhibit the broad teachings of communication system 100 as it may be applied to a myriad of other architectures.

Although the present disclosure has been described in detail with reference to particular arrangements and configurations, these example configurations and arrangements can be varied significantly without departing from the scope of the present disclosure. For example, although the present disclosure has been described with reference to particular communication exchanges involving certain network switching or routing protocols, communication system 100 may be adapted for use with other communication or routing protocols. Moreover, although communication system 100 has been described with reference to particular elements and operations that facilitate the communication process, these elements and operations may be replaced by any suitable architecture or process that achieves the intended functionality of communication system 100.

Numerous other changes, substitutions, variations, alterations, and modifications may be ascertained to one skilled in the art and it is intended that the present disclosure encompass all such changes, substitutions, variations, alterations, and modifications as falling within the scope of the appended claims. To assist the U.S. patent and trademark office (USPTO), and any reader of any patent issued on this application, in interpreting the appended claims, applicants intend to point out that the applicant: (a) it is not intended that any appended claims refer to 35 u.s.c. section 112, paragraph six (6), as it exists on the filing date of this application, unless the word "means for … …" or "step for … …" is explicitly used in a particular claim; and (b) is not intended to limit the disclosure in any way in this specification in any way that is not otherwise reflected in the appended claims.

Claims

1. A method for communication, comprising:

determining a variance of path metrics for a plurality of cellular interfaces;

determining whether there is a difference between the determined variance and a previous variance of the path metrics determined for the plurality of cellular interfaces; and is

Updating a routing table of the plurality of cellular interfaces using the determined variance if there is a difference between the determined variance and the previous variance of path metrics determined for the plurality of cellular interfaces, wherein updating the routing table comprises applying the determined variance to a particular path metric of a particular cellular interface that is the smallest of the path metrics of the plurality of cellular interfaces.

2. The method of claim 1, further comprising:

storing the determined variance in at least one storage of a router if there is a difference between the determined variance and the previous variance of the path metrics determined for the plurality of cellular interfaces.

3. The method of claim 1, further comprising:

providing non peer-to-peer load balancing for the plurality of cellular interfaces according to the routing table.

4. The method of claim 1, wherein a particular path metric for a particular cellular interface is based at least in part on a bandwidth of the particular cellular interface, and wherein the bandwidth is dynamically adjustable based at least in part on throughput through the particular cellular interface.

5. The method of claim 1, wherein determining the variance of path metrics for the plurality of cellular interfaces is performed according to a predetermined time interval.

6. The method of claim 1, wherein determining the variance of path metrics for the plurality of cellular interfaces is performed in response to determining that a bandwidth of a particular cellular interface has been adjusted.

7. The method of claim 1, wherein each of the plurality of cellular interfaces is provided for a router, and wherein at least one of the plurality of cellular interfaces is connectable to a Long Term Evolution (LTE) radio access network.

8. A non-transitory tangible medium encoding logic including instructions for execution, the instructions when executed by a processor operable to perform operations comprising:

determining a variance of path metrics for a plurality of cellular interfaces;

determining whether there is a difference between the determined variance and a previous variance of the path metrics determined for the plurality of cellular interfaces; and

9. The medium of claim 8, the operations further comprising:

10. The medium of claim 8, the operations further comprising:

providing non-peer load balancing for the plurality of cellular interfaces according to the routing table.

11. The media of claim 8, wherein determining the variance of path metrics for the plurality of cellular interfaces is performed in response to determining that a bandwidth of a particular cellular interface has been adjusted, and wherein the bandwidth is dynamically adjustable based at least in part on throughput through the particular cellular interface.

12. An apparatus for communication, comprising:

a memory element for storing data; and

a processor that executes instructions associated with the data, wherein the processor and the memory element cooperate to cause the apparatus to be configured to:

determining a variance of path metrics for a plurality of cellular interfaces;

13. The apparatus of claim 12, wherein determining the variance of a path metric further comprises:

determining a ratio between a minimum path metric of the plurality of cellular interfaces and a maximum path metric of the plurality of cellular interfaces.

14. The apparatus of claim 12, wherein a particular path metric for a particular cellular interface is based at least in part on a bandwidth of the particular cellular interface, and wherein the bandwidth is dynamically adjustable based at least in part on throughput over the particular cellular interface.

15. The apparatus of claim 12, wherein determining the variance of path metrics for the plurality of cellular interfaces is performed in response to determining that a bandwidth of a particular cellular interface has been adjusted.

16. The apparatus of claim 12, wherein the apparatus is a router, and wherein at least one of the plurality of cellular interfaces is connectable to a Long Term Evolution (LTE) radio access network.