US20120057513A1 - Sleep mode selection employing a semi-markov decision process - Google Patents

Abstract

The disclosed subject matter relates to energy efficiency in mobile devices associated with a wireless communications environment. Power Saving Classes (PSCs) aim to reduce the power consumption of mobile devices based on the different types of traffic. Optimizing sleep mode selection schemes so as to maximize the energy efficiency while providing a QoS guarantee can be accomplished with optimization of a theoretical framework based on the semi-Markov Decision Process along with a performance evaluation on the sleep mode operation. A novel Policy Optimization algorithm is capable of finding optimized policies for a selection of different PSCs.

Description

RELATED APPLICATIONS
• This application claims the benefit of U.S. Provisional Application No. 61/380,875, filed 8 Sep. 2010, which is hereby incorporated by reference in its entirety.
TECHNICAL FIELD
• The various embodiments of the subject disclosure relate generally to wireless communications, and more particularly to sleep mode selection for components associated with wireless communications.
BACKGROUND
• Growing numbers of users are wirelessly accessing systems such as the internet and cellular telephone systems. Energy efficiency in mobile devices designed to operate with these wireless access systems remains a critical issue in the design of wireless communication systems. Energy efficiency can be improved by employing lower power schemes, e.g., sleep modes or variations thereof. These lower power schemes can reduce energy consumption by cycling between a higher power consumption mode (e.g., an active state) and a lower power consumption mode (e.g., a reduced activity or inactive state) as will be appreciated by those of skill in the art.
• A drawback to lower power schemes is that they can be associated with degraded communications performance. As an example, spending time in an inactive state can decrease signal reception periods and thus decrease communication performance (e.g., when a device is asleep it generally does not receive or transmit data, etc.) In order to provide a high Quality of Service (QoS), wireless standards can support various power saving schema. Each of these power saving schema can include different sleep times or sleep patterns that can have different balances between energy consumption and device performance. For example, the IEEE 802.16e standard provides at least three unique Power Saving Classes (PSCs) which aim to reduce the power consumption of mobile devices based on different types of anticipated wireless traffic. Conventional selection of the various PSCs in wireless systems can improve energy efficiency but much room remains in optimizing the selection of these PSCs for improved energy efficient sleep mode selection schemes.
SUMMARY
• The following presents a simplified summary of the various embodiments of the subject disclosure in order to provide a basic understanding of some aspects described herein. This summary is not an extensive overview of the disclosed subject matter. It is intended to neither identify key or critical elements of the disclosed subject matter nor delineate the scope of the subject various embodiments of the subject disclosure. Its sole purpose is to present some concepts of the disclosed subject matter in a simplified form as a prelude to the more detailed description that is presented later.
• An embodiment of the presently disclosed subject matter, can include a system that facilitates access to one or more sleep mode selection policies, comprising a semi-Markov decision process (MDP) component, wherein the MDP component accesses at least one wireless system parameter and employs a semi-Markov decision process model in determining the one or more sleep mode selection policies.
• In another embodiment, the disclosed subject matter can be in the form of a method, accessing at least one wireless system parameter, determining at least one optimized sleep mode selection policy, and facilitating access to the at least one determined sleep mode selection policy.
• A further embodiment of the disclosed subject matter includes modeling sleep mode operation of a mobile subscriber station (MSS) as a MDP to facilitate computing performance evaluations of wireless power saving classes associated with the MSS and the wireless communications system, modeling a cost function for a sleep mode behavior of the MSS, and selecting an optimized sleep mode selection policy from a set of sleep mode selection policies for the MSS based in part on minimizing the power consumption of the MSS.
• The following description and the annexed drawings set forth in detail certain illustrative aspects of the disclosed subject matter. These aspects are indicative, however, of but a few of the various ways in which the principles of the various embodiments of the subject disclosure can be employed and the disclosed subject matter is intended to include all such aspects and their equivalents. Other advantages and distinctive features of the disclosed subject matter will become apparent from the following detailed description of the various embodiments of the subject disclosure when considered in conjunction with the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a diagram of a system that can facilitate access to one or more optimized sleep mode selection policies in accordance with an aspect of the subject matter disclosed herein.
• FIG. 2 is a diagram of a system that can facilitate access to one or more optimized sleep mode selection policies in accordance with an aspect of the subject matter disclosed herein.
• FIG. 3 is a schematic diagram of a stochastic controllable Markov model for three power saving classes in accordance with an aspect of the subject matter disclosed herein.
• FIG. 4 is a schematic diagram of operational cycles for three power saving classes in accordance with an aspect of the subject matter disclosed herein.
• FIG. 5 illustrates exemplary simulation results and exemplary numerical results for sleep ratio of a plurality of power saving classes in accordance with an aspect of the disclosed subject matter.
• FIG. 6 illustrates exemplary simulation results and exemplary numerical results for energy cost of a plurality of power saving classes in accordance with an aspect of the disclosed subject matter.
• FIG. 7 illustrates a graphic representation of a geometric interpretation of policy optimization for a plurality of power saving classes in accordance with an aspect of the disclosed subject matter.
• FIG. 8 illustrates exemplary pseudo-code for sleep mode policy selection optimization in accordance with an aspect of the disclosed subject matter.
• FIG. 9 illustrates exemplary sleep mode policy distributions for delay-insensitive wireless traffic in accordance with an aspect of the disclosed subject matter.
• FIG. 10 illustrates exemplary sleep mode policy distributions for delay-stringent wireless traffic in accordance with an aspect of the disclosed subject matter.
• FIG. 11 illustrates exemplary energy consumption for Poisson distributed wireless traffic and non-Poisson distributed wireless traffic in accordance with an aspect of the disclosed subject matter.
• FIG. 12 illustrates an exemplary system including sleep mode selection employing a semi-Markov decision process in accordance with an aspect of the disclosed subject matter.
• FIG. 13 is a plot of exemplary data contrasting timeout schema and decision schema in accordance with aspects of the disclosed subject matter.
• FIG. 14 illustrates a method that facilitates access to determined sleep mode selection policies in accordance with an aspect of the disclosed subject matter.
• FIG. 15 illustrates a method that facilitates access to determined sleep mode selection policies in accordance with an aspect of the disclosed subject matter.
• FIG. 16 illustrates a block diagram of an exemplary electronic device that can facilitate access to determined sleep mode selection policies in accordance with an aspect of the disclosed subject matter.
DETAILED DESCRIPTION
• Energy efficiency in mobile devices remains a critical issue in the design of wireless communication systems. Generally a balance must be achieved between Quality of Service (QoS) in wireless communications and power consumption in wireless devices (e.g., mobile subscriber stations (MSS), user equipments (UEs), etc.) For example, the IEEE 802.16e-2005 standard supports three unique Power Saving Classes (PSCs) which aim to reduce the power consumption of mobile devices based on different types of anticipated traffic. An optimizing sleep mode selection scheme to maximize the energy efficiency of wireless devices while providing a certain QoS guarantee is highly desirable. A theoretical framework based on the semi-Markov Decision Process (MDP) along with a performance evaluation on the sleep mode operation can be employed to generate optimized results. As used herein, the term “optimized' is used inclusively to indicate some level of optimization up to and including, but not limited to, an ideal optimization (e.g., an optimized result can be less optimal than an ideally optimized result). A stochastic optimization problem can be solves by way of a novel Policy Optimization (PO) algorithm, based at least in part on the MDP model, which can determine an optimized sleep mode selection policy. Exemplary numerical and simulation results demonstrate the validity of the disclosed subject matter under different QoS requirements such as the packet delay and energy consumption level. Thus, optimized sleep mode selection can provide improved energy consumption while maintaining desired QoS levels. For simplicity, the presently disclosed subject matter will generally be illustrated within the realm of the IEEE 802.16e mobile system, though it can be applied in a nearly limitless number of wireless systems and schemes as will be readily appreciated by one of skill in the relevant arts, and all such permutations are expressly within the scope of the subject disclosure.
• In the IEEE 802.16e-2005 standard (hereinafter IEEE 802.16e), three types of Power Saving Classes (PSCs) are designed for different types of traffic. The main difference between three PSCs is the way that a sleep window is determined. In short, the sleep windows for type I is incrementally doubled if no traffic is sensed. In contrast, for type II, the sleep windows reminds constant. Moreover, for type III, only one sleep window is applied. According to the IEEE 802.16e standard, PSCs of type I are recommended for Best Effort (BE) and non-realtime (NRT) service, while type II is recommended for the unsolicited granted service (UGS) and realtime variable rate (RT-VR) service connection. Further, PSCs of type III is recommended only for use with multicast or management traffic. Sleep mode in the 802.16e standard has been well recognized as an effective way for discontinuous reception, whereby idle devices power down and turn on their receivers at some future time instant based on traffic arrival to prevent unnecessary power usage.
• The disclosed subject matter is described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments of the subject disclosure. It may be evident, however, that the disclosed subject matter can be practiced without these specific details. In other instances, well-known structures and devices are illustrated in block diagram form in order to facilitate describing the various embodiments of the subject disclosure.
• Turning to the figures, FIG. 1 illustrates a system 100 that can facilitate access to one or more optimized sleep mode selection policies in accordance with an aspect of the subject matter disclosed herein. System 100 can include semi-Markov decision process (MDP) component 110. MDP component 110 can access one or more wireless communications systems parameters. For example, a wireless communications systems parameters can indicate UE loading on base station (BS) resources such as how many carrier channels are free, how many UEs are registers to a BS, percentage of active registered UEs on a BS, etc.
• MDP component 110 can facilitate access to one or more optimized sleep model selection policies. In an aspect, MDP component 110 can similarly facilitate access to derivatives of optimized sleep model selection policies, for example timeout interval values, decision values, etc. In accordance with the disclosed subject matter, MDP component 110 can, in an aspect, determine the one or more optimized sleep mode selection policies. These determinations can be based, at least in part, on the wireless system parameters. Further, these determinations can employ cost metric determinations, policy optimization computations, etc.
• Optimized sleep model selection policies can be employed in selecting or switching among a plurality of sleep schema or PSCs. As a non-limiting example, where a UE is regularly receiving packet data, the UE can employ a first sleep mode, however, when a base station indicates that it is heavily loaded, the UE can select a second sleep mode in anticipation of decreased data throughput. One of skill in the art will appreciate that numerous other examples are readily grasped and that all are within the scope of the present disclosure.
• FIG. 2 is a diagram of a system 200 that can facilitate access to one or more optimized sleep mode selection policies in accordance with an aspect of the subject matter disclosed herein. System 200 can be the same as, or similar to, system 100. System 200 can include MDP component 210. MD)P component 210 can include cost metric component 220. Cost metric component 220 can, in an aspect, facilitate cost determinations. For example, cost determinations can relate to packet delay, energy consumption, sleep ratio, sleep mode switching costs, etc.
• In an aspect, system 200 can include policy optimized (PO) component 230. PO component 230 can facilitate optimization of a policy. PO component 230 can, in an aspect, consider one or more of cost metrics (e.g., those facilitated by cost metric component 220, etc.), QoS requirements, traffic conditions, etc., to optimize one or more sleep mode selection policies. These sleep mode selection policies can be accessed to facilitate reduced energy consumption for a MSS.
• For optimizing sleep mode selection policies, it can be assumed that packet arrival to the system follows the Poisson distribution with average rate λ packets per frame. Without loss of generality, a fixed frame length can be used (for simplicity, the present disclosure uses 1 ms herein throughout unless otherwise expressly or inherently stated). Further, let μ denote the mean packet service rate between a BS and a MSS. For clarity, the discussion herein focuses on a point-to-point downlink communication from a BS associated with one MSS although, as will be appreciated by one of skill in the art, this is not meant to be a limitation of the presently disclosed subject matter.
• Turning now to FIG. 3, presented is a schematic diagram 300 of a stochastic controllable Markov model for three power saving classes in accordance with an aspect of the subject matter disclosed herein. The MDP model employed is illustrated as a transitional diagram, where each state represents one unique sleep stage. According to the MDP definition, the model can be abstracted as a three-state set I={S_N, S_I, S_II}, representing normal sleep mode, sleep mode of type I and sleep mode of type II. FIG. 3 illustrates each of these states respectively joining with some policy (denoted as R) for controlling operation mode. R is defined as a prescription for taking actions at each decision epoch, whether a deterministic policy or a randomized policy. It is assumed that each decision epoch only occurs once a MSS becomes idle (termination of busy period) in the normal mode. The action variable a is chosen from a three-command set A={s_N,s_I, s_II}, with the intuitive meaning of “switching to normal sleep mode”, “switching to sleep mode type I”, and “switching to sleep mode type II”, respectively. To simplify notation herein, we symbolize the three distinctive actions as a_i where I=1, 2, or 3. For illustration, when an action is taken, MSS will move to a new state in the next frame with a probability depending on the action. The model assumes the subsequently chosen action a is independent of past history. Wherein the operational mode switching is not a random behavior, the illustrated transitions (dashed lines in FIG. 3) are responsive to specific actions and are thus deterministic. For example, P[s′]s, a] is 1, meaning the current state s transfers to the next state s′ with probability 1 when a given action is taken.
• Initially, a MSS stays in normal sleep mode if action s_N occurs, transmitting and receiving packets. Otherwise, the MSS goes to PSC of type I immediately if action s_I occurs or the MSS immediately goes to type II if action s_II occurs. To illustrate the type I, let S₁(k) (0≦k≦w) be the multiple sleep stages and w be the final sleep stage indicator. For example, according to the IEEE 802.16e standard, each S₁(k) has a binary-increasing sleep window size 2^kT₀ and a constant listen interval T_L. The MSS receives the traffic indication message from the BS in T_L to decide whether to wake up or remain asleep. When packets arrive at the BS, the BS transmits a positive traffic indication message to the MSS in T_L to notify the arrival (shown in FIG. 4 at 410). As such, p_k is the probability of packet arrival at BS in S_t(k) equal to the transition probability from S_t(k) to S_N. If the MSS gets a negative traffic indication, then it transitions to the next sleep stage S _t(k+1) with probability (1−p_k) until it reaches the final sleep stage S_t(w). Under the Poisson arrival assumption, we have
• 
  p _k=1−e ^{−λV k} ,   (1)
• where V_k is denoted as vacation time at stage k, which is
• 
  V _k=2^k T ₀ +T _L, 0≦k≦w.   (2)
• If action s_II is taken, the MSS switches to sleep mode type II that consists of one single sleep state S_II with constant sleep window size T_S. The MSS can remain in sleep mode if the accumulative arriving packets during the previous vacation time V_II(V_II=T_S+T_L) is less than or equal to the maximum packet number d. As such, the MSS can be allowed to transmit packets during the period of T_L. d=└μ·T_L┘, depending on the length of T_L and transmission rate. Otherwise, MSS can return to the normal sleep mode if the total incoming packets exceed d. Let Q be the transition probability from S_II to S_N such that
• $\begin{array}{cc}Q\stackrel{\Delta }{=}\mathrm{Pr}\left\{J>d\right\}=1-\sum _{j=0}^d\frac{{\left(\lambda V_{\mathrm{II}}\right)}^j{}^{-\lambda V_{\mathrm{II}}}}{j!}& \left(3\right)\end{array}$
• It is possible to find an optimized sleep mode selection policy R* through assigning optimal values among these controllable transitions. A sleep mode selection policy R can be written as the solution set of all controllable probabilities, namely, {x_i(a),∀i∈I, ∀a∈A}, where x_i(a) (0≦x_i(a)≦1) is known as the state-action frequencies. It will be appreciated by one of skill in the art that given the total number of decisions issued in state i , x_i(a) is the expected proportion of times that command a is selected. Thus, x_i(a) satisfies Σ_a∈Ax_i(a)=1 for any given i ∈ I. In the model illustrated at 300, decisions do not occur in sleep mode, which means ∀i∈{S_I, S_II}, x_i(a)=0, ∀a∈A. R becomes a 3-tuple {x_{S N (s N} , x_{S N (s I )}, x_{S N (s II )}}. For a given policy R such that all the x_i(a) are given, there must be a unique equilibrium distributionπ_j ^R, j ∈ I that can be obtained from the following equations
• $\begin{array}{cc}\{\begin{array}{c}{\pi }_{S_I}^{\left(R\right)}\left(k\right)={\pi }_{S_I}^{\left(R\right)}\left(0\right)\prod _{i=0}^{k-1}\left(1-p_i\right),1\le k\le w-1\\ {\pi }_{S_I}^{\left(R\right)}\left(w\right)={\pi }_{S_I}^{\left(R\right)}\left(0\right)\prod _{i=0}^{w-1}\left(1-p_i\right)+{\pi }_{S_I}^{\left(R\right)}\left(w\right)\left(1-p_w\right)\\ {\pi }_{S_N}^{\left(R\right)}={\pi }_{S_N}^{\left(R\right)}x_{S_N}\left(\mathrm{s\_N}\right)+\sum _{k=0}^w{\pi }_{S_I}^{\left(R\right)}\left(k\right)p_k+{\pi }_{S_{\mathrm{II}}}^{\left(R\right)}Q\\ {\pi }_{S_{\mathrm{II}}}={\pi }_{S_N}^{\left(R\right)}x_{S_N}\left(\mathrm{s\_II}\right)+\left(1-Q\right){\pi }_{S_{\mathrm{II}}}^{\left(R\right)}\\ 1=\sum _{k=0}^w{\pi }_{S_I}^{\left(R\right)}\left(k\right)+{\pi }_{S_N}^{\left(R\right)}+{\pi }_{S_{\mathrm{II}}}^{\left(R\right)}\\ 1=\sum _{a\in A}x_{S_N}\left(a\right)\end{array}& \left(4\right)\end{array}$
• where π_{S I} ^(R)(k), k=0,1,2, . . . , w, is the steady state probability for the sleep mode at stage k under policy R, while π_{S II} ^(R) and π_{S N} ^(R) are the steady state probabilities of type II and normal mode, respectively. Since w, T₀, T_s and T_L are with sleep mode configurations and λ, μ are known by given system environment, if R is known, all the steady state probabilities can be obtained.
• Another basic element of the MDP framework includes cost metrics (or cost functions). These cost metrics are relevant to the optimization. As non-liminitng examples, sleep ratio, energy cost, and packet delay can be used to illustrate cost metrics. Let i be the present state associated with each action a taken. For better illustration, let γ _i(a), c _i(a), and d _i(a) be the expected sleep ratio, the expected power consumption level, and the expected packet delay respectively. Whereas decisions only occur in state S_N (as stated previously), we will omit the subscript S_N for notational succinctness. Furhter, let Γ(a), C(a), D(a) be the vector form of each cost metric, respectively:
• $\begin{array}{cc}\Gamma \left(a\right):=\left[\begin{array}{c}\stackrel{\_}{\gamma }\left(\mathrm{s\_N}\right)\\ \stackrel{\_}{\gamma }\left(\mathrm{s\_I}\right)\\ \stackrel{\_}{\gamma }\left(\mathrm{s\_II}\right)\end{array}\right]C\left(a\right):=\left[\begin{array}{c}\stackrel{\_}{c}\left(\mathrm{s\_N}\right)\\ \stackrel{\_}{c}\left(\mathrm{s\_I}\right)\\ \stackrel{\_}{c}\left(\mathrm{s\_II}\right)\end{array}\right]D\left(a\right):=\left[\begin{array}{c}\stackrel{\_}{d}\left(\mathrm{s\_N}\right)\\ \stackrel{\_}{d}\left(\mathrm{s\_I}\right)\\ \stackrel{\_}{d}\left(\mathrm{s\_II}\right)\end{array}\right]& \left(5\right)\end{array}$
• in which each row represents a corresponding action taken from A . The cost metrics can thus be derived from the sleep behavior decribed in FIG. 4. With regard to FIG. 4, illustrated is a set of exemplary, non-limiting, schematic diagrams, 400, 410 and 420, illustrating operational cycles for three PSCs in accordance with an aspect of the subject matter disclosed herein. The three scenarios in FIG. 4 satisfy the following Markovian properties: if an action a is taken at the decision epoch in state i, then the time between now and the next decision epoch depends only on the present state i. Also, the cost functions depend only on the present state and chosen action. Additionally, for any chosen decision, S_N is strictly recurrent [3] since MSS will eventually return to S_N with probability 1. Such property introduces the application of renewal-reward [3] theory, which leads to the following important conclusion: the average cost equals to the long-run average actual cost per time unit when policy R is used independent of the initial state.
• Let N, I and II be the superscripts for the normal sleep mode, type I and type II PSCs, respectively. To measure the usage of serving BS's air interface resource (AIR), the sleep ratio γ can be defined as the time that MSS can power down its transceiver (release its uplink and downlink connections with BS) compared with the total operation time. Thus γ is a system-level performance metric that reflects a BS's connection capacity in its service area. It is evident that γ(s_N)=0, whereas γ(s_I) and γ(s_II) are written as
• $\begin{array}{cc}\stackrel{\_}{\gamma }\left(\mathrm{s\_I}\right)=\frac{{\stackrel{\_}{\tau }}_S^I}{{\stackrel{\_}{\tau }}_V^I},\stackrel{\_}{\gamma }\left(\mathrm{s\_II}\right)=\frac{{\stackrel{\_}{\tau }}_S^{\mathrm{II}}}{{\stackrel{\_}{\tau }}_V^{\mathrm{II}}},& \left(6\right)\end{array}$
• which is the expected sleep time τ _S (excluding T_L, because MSS needs to wake up during the T_L to receive at least one traffic indication message, the same way as in normal mode) divided by the total expected operation cycle time τ _v.
• For the duration of τ_V ^I (see FIG. 4 at 410), MSS transfers to PSC of type I, traverses some sleep stages, and wakes up upon traffic arrival to transmit data until it is idle. Therefore, the τ_V ^I consists of one duration of sleep mode, followed by a busy period B^I to transmit the accumulated data arriving during the prior vacation period. For notational convenience, denote {tilde over (p)}_k as the probability that MSS sleeps for k vacation cycles before switching. Further, k can exceed the value of final sleep stage w, after which MSS remains in S_I(w) for (k−w+1) cycles until it wakes up. Let τ _V ^I(k) be the expected duration after sleeping for k vacations until the next decision epoch. Thus, τ _V ^I can be written as
• $\begin{array}{cc}{\stackrel{\_}{\tau }}_V^I=\sum _{k=0}^{\infty }{\tilde{P}}_k{\stackrel{\_}{\tau }}_V^I\left(k\right),\mathrm{where}{\tilde{p}}_k\mathrm{and}{\stackrel{\_}{\tau }}_V^I\left(k\right)\mathrm{are}& \left(7\right)\\ {\tilde{p}}_k=\{\begin{array}{c}{\mathrm{<figure-callout\; id="0"\; label="p"\; filenames="US20120057513A1-20120308-D00005.png,US20120057513A1-20120308-D00006.png"\; state="\{\{state\}\}">p</figure-callout>}}_0,k=0\\ p_k\prod _{i=0}^{k-1}\left(1-p\right),1\le k\le w-1\\ {p_w\left(1-p_w\right)}^{k-w}\prod _{i=0}^{w-1}\left(1-p\right),k\ge w\end{array}& \left(8\right)\\ {\stackrel{\_}{\tau }}_V^I\left(k\right)=\{\begin{array}{c}\sum _{i=0}^kV_i+E\left[B^I\left(k\right)\right],0\le k\le w-1_k\\ \sum _{i=0}^wV_i+\left(k-w\right)V_w+E\left[B^I\left(w\right)\right],k\ge w\end{array}& \left(9\right)\end{array}$
• E[B^I(k)] is the mean time of exceptional busy period to transmit previously buffered traffic accumulated in sleep stage k. To derive E[B^I(k)], take into account the scenario of new arrival packets during B^I(k). Thus the expected value becomes
• 
  E[B ^I(k)]=(E[M _buf(k)]+E[M _new(k)])E[S],   (10)
• where E[M_buf(k)]=λV_k (Little's Law) is the mean number of packet accumulated during prior vacation period and E└M_new(k)┘=λE└B^I(k)┘ is the mean number of new arriving packet during the B^I(k). S is the random variable of service time for each packet and E[S]=1/μ. Thus, we can obtain
• $\begin{array}{cc}E\left[B^I\left(k\right)\right]=\frac{\lambda V_k}{\mu -\lambda },& \left(11\right)\end{array}$
• Since sleep time only includes binary-increased sleep window, τ _S ^I can be derived using above equations except T_L and busy period. In other words, the pure sleep time for type I is
• 
  τ _V ^I=Σ_k=0 ^∞ {tilde over (P)} _k τ _V ^I(k),   (12)
• where τ _S ^I(k) is the expected sleep time for stage k and
• $\begin{array}{cc}{\stackrel{\_}{\tau }}_S^I\left(k\right)=\{\begin{array}{c}\sum _{i=0}^k2^i{\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="US20120057513A1-20120308-D00005.png,US20120057513A1-20120308-D00006.png"\; state="\{\{state\}\}">\; <figure-callout\; id="0"\; label="T"\; filenames="US20120057513A1-20120308-D00005.png,US20120057513A1-20120308-D00006.png"\; state="\{\{state\}\}">T</figure-callout>\; </figure-callout>}}_0,0\le k\le w-1\\ \sum _{i=0}^w2^i{\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="US20120057513A1-20120308-D00005.png,US20120057513A1-20120308-D00006.png"\; state="\{\{state\}\}">T</figure-callout>}}_0+\left(k-w\right)2^w{\mathrm{<figure-callout\; id="0"\; label="T"\; filenames="US20120057513A1-20120308-D00005.png,US20120057513A1-20120308-D00006.png"\; state="\{\{state\}\}">T</figure-callout>}}_0,k\ge w,\end{array}& \left(13\right)\end{array}$
• which is equivalent to the vacation time τ _V ^I excluding the listen interval and busy period.
• For type II (see FIG. 4 at 420) with single constant vacation cycle, the duration of one operational cycle can be obtained according to the geometric distribution:
• $\begin{array}{cc}{\stackrel{\_}{\tau }}_V^{\mathrm{II}}=\sum _{k=1}^{\infty }\left\{kV_{\mathrm{II}}+E\left[B^{\mathrm{II}}\right]\right\}{\left(1-Q\right)}^{k-1}Q=\frac{1}{Q}V_{\mathrm{II}}+E\left[B^{\mathrm{II}}\right]& \left(14\right)\end{array}$
• where E[B^II]=λV_II/(μ−2) and (1−Q)^k−1Q indicates the probability of k successful consecutive sleep periods before waking up. The corresponding sleep time is τ _S ^II=T_S/Q.
• To evaluate the power usage of MSSs the energy cost c can be derived wherein it is intuitively interpreted as the mean power consumption level of the system. Denote P_B, P_I, P_S, P_L as the given power consumption level in busy period, idle period, sleep time, and listen interval, respectively. Whereas the normal mode time alternates between idle periods I (no arrival of packets) and busy period B (at least one packet arrival in transmission). Energy cost c(s_N) is then expressed as
• $\begin{array}{cc}\stackrel{\_}{c}\left(\mathrm{s\_N}\right)=\frac{P_IE\left[I\right]+P_BE\left[B\right]}{E\left[I\right]+E\left[B\right]},& \left(15\right)\end{array}$
• and the average idle time and the average busy period can be expressed as
• $\begin{array}{cc}E\left[I\right]=\frac{1}{\lambda },E\left[B\right]=\frac{E\left[S\right]}{1-\lambda E\left[S\right]},& \left(16\right)\end{array}$
• For type I, the mean power consumption level is written as
• $\begin{array}{cc}\stackrel{\_}{c}\left(\mathrm{s\_I}\right)=\frac{E_{\mathrm{SW}}=\sum _{k=0}^{\infty }{\tilde{p}}_k{\stackrel{\_}{\varepsilon }}^I\left(k\right)}{{\stackrel{\_}{\tau }}_V^I},& \left(17\right)\end{array}$
• which is the total expected energy consumption of one regenerative cycle, divided by the operation cycle time. E_sw denotes the energy consumption switching between normal mode and sleep mode (activation and deactivation). ε ^I(k) is the total expected energy consumption when system wakes up after k vacation cycles. It is given by
• $\begin{array}{cc}{\stackrel{\rightharpoonup }{\varepsilon }}^I\left(k\right)=\{\begin{array}{c}\sum _{i=0}^k{\varepsilon }_V^I\left(i\right)+P_BE\left[B^I\left(k\right)\right],0\le k\le w-1\\ \sum _{i=0}^w{\varepsilon }_V^I\left(i\right)+\left(k-w\right){\varepsilon }_V^I\left(w\right)+P_BE\left[B^I\left(w\right)\right],k\ge w\end{array}& \left(18\right)\end{array}$
• in which ε_V ^I(i)=P_S2ⁱT₀+P_LT_L (0≦i≦w) is the energy consumption at sleep stage i.
• Whereas type II can take advantage of listen interval to transmit limited arriving packets, a random busy period B_L during T_L is more likely to exist. By applying Little's Law, the mean value of such busy time is E[B_L]=λV_II/μ. Let ε_V ^II be the whole energy consumption during the vacation period of type II:
• $\begin{array}{cc}{\varepsilon }_V^{\mathrm{II}}=\left(\frac{1}{Q}-1\right)\left\{P_sT_s+P_L\left(T_L-E\left[B_L\right\}\right)+P_BE\left[B_L\right]\right\}+\left(P_ST_S+P_LT_L\right),& \left(19\right)\end{array}$
• which consists of (1/Q−1) replicated vacation cycles (scenario illustrated in FIG. 2( c) the 1^st and 2^nd vacation cycles) and one last cycle before waking up (the 3^rd vacation cycle). In Eq.(19), we assume the power level of busy period in T_L is same as P_B. c(s_II) can be written as
• $\begin{array}{cc}\stackrel{\_}{c}\left(\mathrm{s\_II}\right)=\frac{E_{\mathrm{SW}}+{\varepsilon }_V^{\mathrm{II}}+P_BE\left[B^{\mathrm{II}}\right]}{{\stackrel{\_}{\tau }}_V^{\mathrm{II}}}.& \left(20\right)\end{array}$
• As shown in FIG. 4. (at 410 and 420), incoming traffic can experience a delay when the MSS is still asleep. The illustrated MDP model, it is given that the packet arrival follows the Poisson distribution. However, the service rate can fall into the general distribution mainly due to the irregularity of sleeping periods, which depends on the types of sleep mode. Thus, M/G/1 queue is the best option to describe the packets arrival and departure. Therefore, d(s_N) is the direct result from M/G/1 model,
• 
  d (s _N)=E[W]+E[S],   (21)
• where E[W] is the expected waiting time of the packet. Based on Pollaczek-Khintchine (P-K) mean value formula [5], E[W] is defined as
• $\begin{array}{cc}E\left[W\right]=\frac{\rho E\left[R\right]}{1-\rho },& \left(22\right)\end{array}$
• where ρ@λ/μ is traffic intensity and E[R]=E[S²]/(2E[S]) is the residual processing time.
• To derive packet delay under sleep mode, the expected packet delay d _S when the server is on vacation can be calculated as
• 
  d _S = d _V + d _Q + d (s _N),   (23)
• in which the first term is the average remaining vacation time for the packet. d _Q is the expected queuing delay after MSS wakes up and transmits the accumulative packet arrived during vacation, followed by the mean packet delay in normal mode.
• According to M/G/1 with server vacation, d _V is
• $\begin{array}{cc}{\stackrel{\_}{d}}_V=\frac{E\left[V^2\right]}{2E\left[V\right]},& \left(24\right)\end{array}$
• where the random variable V is the duration of vacation time. For types I and II, vacation time is a fixed value depending on the PSC type and the sleep stage, and E[V]=V, E[V²]=V².
• After waking up, the system performs data transmission and empties the buffer, where each packet endures a queuing delay since the possibility of more than one packet arriving during the same V is high. Therefore, d _Q indicates the delay of transmitting preceding packets (relative to a given packet) in the buffer and can be calculated as
• $\begin{array}{cc}{\stackrel{\_}{d}}_Q=\frac{\mathrm{<figure-callout\; id="2"\; label="V"\; filenames="US20120057513A1-20120308-D00003.png,US20120057513A1-20120308-D00005.png"\; state="\{\{state\}\}">V</figure-callout>}}{2}\rho ,& \left(25\right)\end{array}$
• Eq.(25) can be derived by using batch-arrival and server-vacation queuing model. Due to the space limit, the derivation has been omitted. Putting Eqs.(24), (25) and (21) into (23), we obtain the exact expression of d _S.
• For type I, let d_S ^I(k) be the mean sleep delay for sleep stage k. Rewrite the expression of d _s:
• $\begin{array}{cc}{\stackrel{\_}{d}}_{\mathrm{<figure-callout\; id="1"\; label="S"\; filenames="US20120057513A1-20120308-D00003.png,US20120057513A1-20120308-D00004.png"\; state="\{\{state\}\}">S</figure-callout>}}^1\left(k\right)=\frac{{\mathrm{<figure-callout\; id="2"\; label="V\; k"\; filenames="US20120057513A1-20120308-D00003.png,US20120057513A1-20120308-D00005.png"\; state="\{\{state\}\}">V</figure-callout>}}_k}{2}\left(1+\rho \right)+\stackrel{\_}{d}\left(s_N\right),0\le k\le w.& \left(26\right)\end{array}$
• For packet-level delay, the process that packet arrival can be in any one of the sleep stage is well recognized as semi-Markov processes. Thus, define P^I(k) as the proportion of time the packet is in sleep stage k. It is given by
• $\begin{array}{cc}{P}^I\left(k\right)=\frac{{\pi }_{S_I}^{\left(s\_I\right)}\left(k\right)V_k}{\sum _{j=0}^w{\pi }_{S_I}^{\left(s\_I\right)}\left(j\right)V_j},0\le k\le w,& \left(27\right)\end{array}$
• which should be a weighted average of the π_{S I} ^{(s — I)}(k) multiplied by the mean sojourn time spent on sleep stage k, divided by the whole process time. π_{S I} ^{(s — I)}(k), k=0,1,2, . . . , w, is the steady state probability for the sleep mode at stage k when deterministic command s_I is taken. Thus, d(s_I) is
• 
  d (s _— I)=Σ_k=0 ^w d _S ^(I)(k)P ^I(k).   (28)
• Similarly, the packet delay of type II is
• $\begin{array}{cc}\stackrel{\_}{d}\left(\mathrm{s\_II}\right)=\frac{v_{\mathrm{II}}\left(1+\rho \right)}{2}+\stackrel{\_}{d}\left(\mathrm{s\_N}\right).& \left(29\right)\end{array}$
• FIG. 5 illustrates exemplary simulation results and exemplary numerical results 500 for sleep ratio of a plurality of power saving classes in accordance with an aspect of the disclosed subject matter. the results of our evaluation based on the proposed cost metrics. Simulation and numerical results 500 are derived from validating the analytical MDP model bee against a discrete event simulation model built upon a Java Network Simulator (JNS), which is an implementation of the ns-2 simulator with trace files production (same format as NAM trace files). The exemplary simulator parameter settings are set in according with the IEEE 802.16e standard configurations (see sections 6.3.2.3.44 and 10.1 of the IEEE 802.16e-2005 standard). The generated traffic in the exemplary simulation follows a Poisson distribution. The parameters in our numerical calculations and simulations are listed in Table I. Further, whereas the power consumption information of the IEEE 802.16e standard is not currently specified, assume that the power consumption values are in each state as follows: P_B=750 mW, P_I=P_L=170 mW and P_s=50 mW, and for switching cost E_sw=1×10⁻³ J per switch behavior.
• More specifically, FIG. 5 illustrates exemplary simulation results and exemplary numerical calculations on sleep ratio against the traffic load for PSCs of type I and type II. These two curves reveal the key features of different sleep patterns between two PSCs: For type I, MSS inclines to enter deep sleep under light traffic, while for type II it has to wake up periodically to receive indication message so as to maintain responsive even when Σ=0.

• TABLE 1
  Exemplary Numerical and Simulation Parameter Settings
  (see FIGS. 5 and 6).

  	Parameters	Settings

  Payload Size

  	500	bytes
  	MAC Frame Duration	1	ms
  	Propagation Delay
  	1	ms
  	Sleep Stage w (Type I)	9
  	Listening Interval Size	1	Frame
  	Initial Sleep Window Size (Type I)	1	Frame
  	Final Sleep Window Size (Type I)	512	Frames
  	Sleep Window Size (Type II)	1	Frame

• FIG. 6 illustrates exemplary simulation results and exemplary numerical results 600 for energy cost of a plurality of power saving classes in accordance with an aspect of the disclosed subject matter. FIG. 6 is obtained under the same parameters as those given for FIG. 5. Looking more closely at, FIG. 6, the exemplary energy cost on mean power consumption in full normal mode, for PSCs of types I and II is illustrated. The exemplary energy cost grows linearly with traffic load without sleep mode operation (simulation plot is dashed line with diamonds). It can furhter be observed that for types I and II some benefits on energy saving can be achieved. Type I (simulation plot is dashed line with “x”) achieves better energy performance under light traffic mainly due to its deep sleep characteristics. For type II (simulation plot is dashed line with “.”) performes best in the range of medium traffic situation, partly because of its capability of transmitting data during listen interval. However, when traffic grows more intensive, types I and II suffer frequent sleeping and waking up so that switching cost becomes an enormous overhead. Thus, pure normal mode will be the best policy for power saving in this situation.
• It is evident from that there is a tradeoff between energy consumption and packet delay among operational modes of wireless communications systems, as illustrated with regard to the IEEE 802.16e standard. Therefore, it is clearly desireable to optimize a sleep mode selection policy such that the subsequent transitions between operational modes helps to minimize overall energy consumption while still providing a sufficient level of QoS. Policy optimization (PO) can be based on the above discussed MDP model framework. For clarity and brevity, only two typical energy-efficient design scenarios for a practical IEEE 802.16e system are disclosed though one of skill in the art will appreciate that many other scenarios exist and are within the scope of the present disclosure.
• As already disclosed above, systems can be described by a controlled semi-Markov chain with undetermined policy R. By searching the space of R an optimal R* can be found (e.g., an optimal policy that can approach and/or meet a minimum/maximum cost under a specified level of QoS). Many algorithms are available to solve policy optimization problems efficiently. Some well-known techniques include policy improvement, successive approximations, and linear programming (LP). For ease of illustration LP is selected as an appropriate example technique because it offers a convenient way to handle Markovian decision problems with probabilistic constraints. The reason for using probabilistic constraints is that constraints are normally difficult to estimate or be imposed on certain state frequencies in real-life telecommunication environments. These constraints often involve many random parameters such as traffic demand, MSS demographic data, and transmission rate in wireless channel.
• Further, wherein IEEE 802.16e supports two different services, delay-sensitive and delay-tolerant, two probabilistic constrained PO problems can be formulated for each group separately. In an effort to be concise, only the delay-sensitive problem is examined although one of skill in the art will readily appreciate that the subject disclosure is just as readily applicable to delay-tolerant schemes. For delay-sensitive service, optimization can be viewed as a power consumption minimization issue subject to the delay constraint, i.e.,
• 
  min C^T(a)x(a),   (30)
• such that Σ_a∈Ax_{S N} (a)=1, D^T(a)x(a)≦δ, x_{S N} (a)≧0, ∀a ∈ A, where x(a)=[x_{S N} (s_N) x_{S N} (s_I) x_{S N} (s_II)]^T is the vector form of state action frequencies. Eq. 30 can be solved to minimize the long-run average energy cost per time unit and also require the maximal expected delay to be bounded by a probabilistic constraint δ. The rationale behind Eq.30 is that every given action must associate with certain rewards or penalties (e.g., { γ, c, d}). The allocating fraction in x(a) will thereby weight these cost metrics so as to meet specified parameters, resulting in the long-term behavior of the system,
• For delay-insensitive services, minimizing MSS's energy consumption and simultaneously reduce the AIR occupancy to the benefit of the whole system is desireable. For example, if BS has N available subcarriers (or subchannels) allocated to M MSSs in the IEEE 802.16e Point-to-Multipoint (PMP) system, then based on the current call admission and cell loading status, BS should centrally regulate cell loading and provide the call admission control (CAC) efficiently. If N≧M, BS is capable of allocating at least one dedicated subcarrier to each MSS. However, where the saturation occurs (e.g., N<M), some MSSs might be denied service due to the AIR constraint. Through sleep mode support, portions of AIR can be resued (called virtual AIR) when MSS is on vacation and some saturation can be tolerated. For example, where M is the expected number of active MSSs (i.e., not in sleep mode).a saturated scenario can be represented as
• 
  min C^T(a)x(a),   (31)
• such that Σ_a∈A x_{S N} (a)=1, 1−Γ^T(a)x(a)≦N/ M, x_{S N} (a)≧0, ∀a ∈ A.
• Solving Eqs.30 and 31 can be accomplished by employing the simplex method, though this can be computationally intensive and may be burdensome for a resource-constrained MSS in a runtime environment. However, where the action states are reasonably small (e.g., small enough to search through all of them and find the optimal set of policy) the computation can be more easily achieved. Therefore, a light-weight algorithm can be emplyoed, where derived from the presently disclosed our PO structure.
• FIG. 7 illustrates a graphic representation 700 of a geometric interpretation of policy optimization for a plurality of power saving classes in accordance with an aspect of the disclosed subject matter. From Eq.30, the equation and constraints can be represented geometrically as illustrated in FIG. 7. From this figure, each exterior point represents the action frequency ratio between 0 and 1. Due to its sparse constraints, no interior-point solutions can possibly exist in the problem. Instead, all feasible solutions are located on the vertexes of the quadrangular plane.
• FIG. 8 illustrates exemplary pseudo-code 800 for sleep mode policy selection optimization in accordance with an aspect of the disclosed subject matter. Where there are sparse constraints (e.g., where no interior-point solutions can possibly exist in the problem and all feasible solutions are thus located on the vertexes of the quadrangular plane as in FIG. 7), an efficient solving algorithm for our PO problems can readily be formed, e.g., the pseudo-code 800 illustrated in FIG. 8. Solutions can thus be classifyed into three categories according to the geometric interpretation (see FIG. 7). For notational convenience, subscripts S_N are omitted in the elements of x(a), the state action frequencies. Further, a_i denotes the i^th (i=1,2,3) action variable in A and executing a maximization function (or minimization function) on a vector to find the maximal (minimal) element in that vector can be conducted.
• By way of further explaining exemplary pseudo-code 800, lines 1-4 of code 800, indicate the infeasible solution (no intersection between the equality plane and the halfspace of delay inequality plane). Thus, even if MSS is never put to sleep, it is impossible to achieve a delay below δ (under these conditions simply find a policy that gives a minimal delay in order to meet the requirement as much as possible). Lines 5-8 of code 800, denote a deterministic policy schema (rectangular points), where δ becomes the slack constraint. The required delay can be presummed regardless of the choice of action. Therefore, a system should pick the minimal cost to optimize the energy performance.
• Continuing, lines 9-38 of code 800, correspond to randomized policies (e.g., circle points located at the edge of the cross-section lines). Based on the three different cases of delay bounds, it is desireable to search for the action frequency combination which gives the minimum energy cost. As such, lines 13-17 of code 800, search for a randomized policy when δ lies in between any two delay bounds and the delay is inversely proportional to the energy cost; lines 19-21 are to consider the case of choosing the action that gives the minimal cost when the delay is proportional to the energy cost. Lines 23-26 are similar to lines 5-8 when the delay constraint can be met by any of two selected actions. Lines 28-30 arbitrarily generate a temporary solution if the delay constraint cannot be satisfied by either of the two actions. At the end of each inner for-loop, the exemplary algorithm updates the current combination with the lowest cost combination. Interestingly, most of the processing in Algorithm 1 concentrates on the code between lines 9-38, wherein the searching complexity is O(|A|²−|A|), which is quadratically proportional to |A|, the size of A.
• Traffic load, QoS requirements and distinct targets impact optimal policy distribution. FIG. 9 illustrates exemplary sleep mode policy distributions 900 for delay-insensitive wireless traffic in accordance with an aspect of the disclosed subject matter. In an aspect, where no delay constraint is presented (e.g., a delay-insensitive condition) the deterministic policies can be employed to minimize energy cost under various traffic load wherein it can be expected that the energy cost will follow the lower bound of the three curves in FIG. 6 at 600
• In contrast, FIG. 10 presents exemplary sleep mode policy distribution plot 1000 for delay-stringent wireless traffic in accordance with an aspect of the disclosed subject matter. Where a delay-stringent service is presented, randomized policies between s_II and s_N are observed to the left of infeasible solution region in plot 1000. Choosing the decision in a random way rather than in a deterministic fashion can be the best strategy in terms of resource allocation as well as optimality as disclosed hereinabove. For example, a solution to Eq.30, which provides the relation between a low attainable expected energy consumption c* and fulfillment of a specified packet delay δ can be expressed in a functional form as δ≧g( c*), wherein {( c*δ) s.t. δ≧g( c*)} is the set of feasible allocations. For example, a deterministic policy scenario, constraints must satisfy δ>g( c*) strictly, meaning the resource has not yet been fully utilized and optimality has not been reached. However, where the system yields a randomized policy (e.g., when δ=g( c*)) then {( c*,δ) s.t. δ=g( c*)} can be called the set of efficient allocations. Pairs ( c*,δ) correspond to Pareto points of the power-performance tradeoff curve, namely, they represent solutions of PO that cannot be improved upon in both directions (energy and delay performance), producing Pareto optimality of the system. Thus, randomized policies are appropriate for delay-stringent schema.
• The state-action frequency distribution thus provides information that can be employed in determining when a MSS should choose each of the available different sleep modes. Employing Network Simulator 2 (ns-2) the energy efficiency can be illustrated (e.g., ns-2 simulation of MDP demonstrates the performance of the state-action frequency distribution scheme disclosed hereinabove in selecting directed or randomized sleep mode selection policies.
• FIG. 11 illustrates exemplary energy consumption 1100 for Poisson distributed wireless traffic and non-Poisson distributed wireless traffic in accordance with an aspect of the disclosed subject matter. The simulation can be conducted using ns-2 (e.g., with a conventional trafffic estimator determining λ first) following the settings previously disclosed for the previous models to preserve the consistency. Given a simulation time of 400 seconds and a total numbers of run of 1000, the energy consumption can be averaged over these 1000 runs, such that the 95% confidence interval can be obtained. The exemplary simulation employs simulated voice traffic employing the GSM 6.10 codec represented as the common on/off model. Data traffic is based on Poisson arrival. The exemplary simulation, presents only downlink traffic for simplicity and clarity though the disclosure also applies to uplink traffic as will be appreciated by one of skill in the art. Normalized traffic intensity is estimated through a traffic estimator.
• The exemplary mean energy consumption using simulations for both Poisson and non-Poisson traffic is illustrated at 1100. The 95% confidence interval for both results are the the same as, or similar to, each other. Thus, the algorithm can be considered to perform well when adapting to different kinds of traffic. As such, the MDP model and PO solving algorithm, while based on Poisson distribution, also function well for non-Poisson traffic. It can be observes from the FIG. 11, that the performance of the two kinds of traffic can be the same, or similar. This significant result, as will be appreciated by one of skill in the art, illustrates that the MDP modeling approach is applicable to many kinds of wireless traffic.
• With regard to FIG. 12, an exemplary system 1200 adapted to use sleep mode selection employing a semi-Markov decision process in accordance with an aspect of the disclosed subject matter. Exemplary system 1200 can include MDP module 1210. MDP module 1210 can the same as, or similar to, MDP component 110 or 210. MDP module 1210 can include a cost metric construction component 1212 that can access network device 1213 parameters (e.g., power specifications, etc.) MDP module 1210 can include policy optimization (PO) solver 1214 that can access cost metric related data by way of cost metric construction component 1212. Po solver 1214 can further access additional data, for example QoS requirement 1215, among others (not illustrated). Moreover PO solver can access wireless traffic related data 1234, for example, traffic load data from a BS 1230. BS 1230 can access traffic demand related data 1232 (e.g., λ, etc.) and AIR data 1233 (e.g., cell loading status, etc.)
• PO solver 1214 can facilitate access to one or more optimized sleep mode selection policies, R′ 1216. Whereas the selection policy can comprise at least one of timeout-value or policy decision value, MDP module 1210 can further include a switch 1217. Switch 1217 can be any physical or logical switch or combinations thereof. Policy decision values can be passed to policy releaser 1218 that can facilitate access to the decision value or derivatives thereof 1250 by MSS 1211. Timeout-values can be passed to timeout component 1240 which can facilitate access to the timeout-values or derivatives thereof (e.g., a) by MSS 1211. MSS 1211 can provide BS 1230 with access to a mode selection policy 1235.
• In an aspect, FIG. 12 illustrates an approach to implement sleep mode operation in a runtime environment for a MSS 1211 (or UE). MSS 1211 can, by way of MDP module 1210) initially compute and build up the cost metrics based on the system information and its own parameter settings; further the cost metrics, QoS constraints, and optimization targets can be passed to the PO solver for computing an optimized sleep mode selection policy. The output can be a set of state-action frequencies (e.g., at 1216) that can be translated into command probabilities (e.g., by the policy releaser 1218). The policy releaser 1218, in an aspect, can be responsible for issuing decisions (e.g., 1250) that are either randomized or deterministic (as previously disclosed herein) at each decision epoch. In a further aspect, policy R* 1216 can be related the optimized timeout value a that triggers transitions between sleep modes. For example, where MSS 1211 initially stays in S_N and sleeps after it has been idle (no incoming traffic) for a certain amount of time α. If α expires before traffic shows up, MSS 1211 switches to S_I or S_II (e.g., depending on the value of x_{s N} (s_I) and x_{S N} (s_II)). If a does not expire before traffic shows up, MSS 1211 can remain in S_N with probability (1−e^−λα), which can be regarded as equivalent to the transition probability of s_N (see FIG. 3). Where, 1−e^−λα=x_{S N} (s_N)), it follows that
• $\begin{array}{cc}\alpha \stackrel{\Delta }{=}\frac{1}{\lambda }\ln \frac{1}{1-x_{s_N}\left(\mathrm{s\_N}\right)},& \left(32\right)\end{array}$
• which is a function of state action frequency x_{S N} (s_N) and traffic demand λ. Where x_{S N} (s_N)=0, α becomes 0, indicating an instant switching scenario.
• FIG. 13 is a plot 1300 of exemplary data contrasting timeout schema and decision schema in accordance with aspects of the disclosed subject matter. Simulations showing that “timeout-values” (e.g., a values from FIG. 12) result in results that are the same as, or similar to, policy decision results (e.g., FIG. 12 at 1250). Two simulations are illustrated for timeout scheme and PR approach respectively. Both run for 10 minutes and parameter settings follow Table 1 as presented hereinabove. The total number of sleep mode switches and accumulative duration for various states are tracked in the simulation. The PO formulation is based on Eq.(30), where δ=2.3 ms. Two traffic conditions are employed (λ=0.05 and 0.33). For λ=0.05, x(a)=[0.4787 0 0.5213]^T that can be approximated as equally switching between S_N and S_II, while the timeout value can be calculated by Eq.(32): α≈13.0 ms. For λ=0.33, x(a)=[0.9192 0 0.0808]^T and α≈7.5 ms. The time distributions are plotted such that PR and timeout values are compared by pair. The comparison illustrates similar results for both PR and timeout values for the total time staying in different stages (idle, busy, and vacation). Further, the frequencies of entering the sleep mode for both schemes are nearly the same(not illustrated), 14773 total entry times for λ=0.05 and 11241 for λ=0.33. The simulation demonstrates that the PR approach and timeout schemes can produce the same performance results.
• FIGS. 14-15 illustrate methods and/or flow diagrams in accordance with the disclosed subject matter. For simplicity of explanation, the methods are depicted and described as a series of acts. It is to be understood and appreciated that the various embodiments of the subject disclosure is not limited by the acts illustrated and/or by the order of acts, for example acts can occur in various orders and/or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts may be required to implement the methods in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the methods could alternatively be represented as a series of interrelated states by way of state diagram or events. Additionally, it should be further appreciated that the methods disclosed hereinafter and throughout this specification are capable of being stored on an article of manufacture to facilitate transporting and transferring such methods to computers. The term article of manufacture, as used herein, can encompass a computer program accessible from any computer-readable device, carrier, or media.
• FIG. 14 illustrates a method 1400 that facilitates access to determined sleep mode selection policies in accordance with an aspect of the disclosed subject matter. At 1410, method 1400 can include accessing one or more wireless system parameter(s). The parametric information can include wireless traffic conditions. At 1420, an optimized sleep mode selection policy can be determined. In an aspect, this determination can be based, at least in part, on the accessed wireless system parameter of 1410. Determining the selection policy can include employing a semi-Markov decision process (MDP) model. This model can be the same as, or similar to, that disclosed herein, for example, with regard to FIG. 3, etc.
• In a further aspect, the determination at 1420 can include cost metric evaluation. Cost metric evaluation can include determining cost metrics for sleep ratio, energy cost, packet delay, etc. Cost metric evaluation can be the same as, or similar to, that disclosed herein, for example, in regard to FIGS. 4-6, among others. Determining the long term costs of transitions between sleep modes can facilitate quantitative determinations of the lowest power consumption over time while still meeting minimum performance conditions.
• Determining at 1420 can further include policy optimization. That is, where there are more than one possible permutations of sleep mode selection policies, the policies can be optimized to seek a best case policy schema. Policy optimization can include determinations of power minimization, though policy optimization as it is disclosed herein is not so limited. For example, policy optimization can be the same as, or similar to, that disclosed with regard to FIGS. 7-8, etc.
• At 1430, access to the determined policy can be facilitated. At this point method 1400 can end. In an aspect, the determined policy can be employed in executing a decision to follow a particular sleep mode at a decision epoch. For example, decision 1250 of FIG. 12 can be related to MSS 1211 selecting a particular sleep mode. In another aspect, the determined policy can be employed in selecting a sleep interval value at a decision epoch. For example, the timeout period a associated with timeout component 1240 of FIG. 12, can be selected or determined from accessing the policy determined at 1420.
• FIG. 15 illustrates a method 1500 that facilitates access to determined sleep mode selection policies in accordance with an aspect of the disclosed subject matter. At 1510, at least one wireless system parameter can be accessed, including at least on one of expected sleep ration, expected power consumption level m, or expected packet delay. The access at 1510 can be the same as, or similar to, that at 1410 of method 1400. At 1520, an optimized sleep mode selection policy determination can occur. The determining at 1520 can include an MDP model and cost metric evaluation. At 1530 a timeout value or transition decision value can be generated based, at least in part, on the determined sleep mode selection policy. At 1540, access to the timeout value or transition decision value can be facilitated. At this point method 1500 can end.
• Referring to FIG. 16, illustrated is a block diagram of an exemplary, non-limiting electronic device 1600 that can utilize determined sleep mode selection policies in accordance with an aspect of the disclosed subject matter. The electronic device 1600 can include, but is not limited to, a computer, a laptop computer, network equipment (e.g. routers, access points), a media player and/or recorder (e.g., audio player and/or recorder, video player and/or recorder), a television, a smart card, a phone, a cellular phone, a smart phone, an electronic organizer, a PDA, a portable email reader, a digital camera, an electronic game (e.g., video game), an electronic device associated with digital rights management, a Personal Computer Memory Card International Association (PCMCIA) card, a trusted platform module (TPM), a Hardware Security Module (HSM), a set-top box, a digital video recorder, a gaming console, a navigation system or device (e.g., global position satellite (GPS) system), secure memory devices with computational capabilities, devices with a tamper-resistant chip(s), an electronic device associated with an industrial control system, an embedded computer in a machine (e.g., an airplane, a copier, a motor vehicle, a microwave oven), and the like.
• Components of the electronic device 1600 can include, but are not limited to, a processor component 1602, a system memory 1604 (with nonvolatile memory 1606), and a system bus 1608 that can couple various system components including the system memory 1604 to the processor component 1602. The system bus 1608 can be any of various types of bus structures including a memory bus or memory controller, a peripheral bus, or a local bus using any of a variety of bus architectures.
• Electronic device 1600 can typically include a variety of computer readable media. Computer readable media can be any available media that can be accessed by the electronic device 1600. By way of example, and not limitation, computer readable media can comprise computer storage media and communication media. Computer storage media can include volatile, non-volatile, removable, and non-removable media that can be implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, nonvolatile memory 1606 (e.g., flash memory), or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by electronic device 1600. Communication media typically can embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
• The system memory 1604 can include computer storage media in the form of volatile and/or nonvolatile memory 1606. A basic input/output system (BIOS), containing the basic routines that help to transfer information between elements within electronic device 1600, such as during start-up, can be stored in memory 1604. Memory 1604 can typically contain data and/or program modules that can be immediately accessible to and/or presently be operated on by processor component 1602. By way of example, and not limitation, system memory 1604 can also include an operating system, application programs, other program modules, and program data.
• The nonvolatile memory 1606 can be removable or non-removable. For example, the nonvolatile memory 1606 can be in the form of a removable memory card or a USB flash drive. In accordance with one aspect, the nonvolatile memory 1606 can include flash memory (e.g., single-bit flash memory, multi-bit flash memory), ROM, PROM, EPROM, EEPROM, and/or NVRAM (e.g., FeRAM), or a combination thereof, for example. Further, the flash memory can be comprised of NOR flash memory and/or NAND flash memory.
• A user can enter commands and information into the electronic device 1600 through input devices (not illustrated) such as a keypad, microphone, tablet or touch screen although other input devices can also be utilized. These and other input devices can be connected to the processor component 1602 through input interface component 1610 that can be connected to the system bus 1608. Other interface and bus structures, such as a parallel port, game port or a universal serial bus (USB) can also be utilized. A graphics subsystem (not illustrated) can also be connected to the system bus 1608. A display device (not illustrated) can be also connected to the system bus 1608 via an interface, such as output interface component 1612, which can in turn communicate with video memory. In addition to a display, the electronic device 1600 can also include other peripheral output devices such as speakers (not illustrated), which can be connected through output interface component 1612.
• It is to be understood and appreciated that the computer-implemented programs and software can be implemented within a standard computer architecture. While some aspects of the disclosure have been described above in the general context of computer-executable instructions that may run on one or more computers, those skilled in the art will recognize that the technology also can be implemented in combination with other program modules and/or as a combination of hardware and software.
• Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices (e.g., PDA, phone), microprocessor-based or programmable consumer electronics, and the like, each of which can be operatively coupled to one or more associated devices.
• As utilized herein, terms “component,” “system,” “interface,” and the like, can refer to a computer-related entity, either hardware, software (e.g., in execution), and/or firmware. For example, a component can be a process running on a processor, a processor, an object, an executable, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and a component can be localized on one computer and/or distributed between two or more computers.
• Furthermore, the disclosed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. For example, computer readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips . . . ), optical disks (e.g., compact disk (CD), digital versatile disk (DVD) . . . ), smart cards, and flash memory devices (e.g., card, stick, key drive . . . ). Additionally it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN). Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the disclosed subject matter.
• Some portions of the detailed description may have been presented in terms of algorithms and/or symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and/or representations are the means employed by those cognizant in the art to most effectively convey the substance of their work to others equally skilled. An algorithm is here, generally, conceived to be a self-consistent sequence of acts leading to a desired result. The acts are those requiring physical manipulations of physical quantities. Typically, though not necessarily, these quantities take the form of electrical and/or magnetic signals capable of being stored, transferred, combined, compared, and/or otherwise manipulated.
• It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the foregoing discussion, it is appreciated that throughout the disclosed subject matter, discussions utilizing terms such as processing, computing, calculating, determining, and/or displaying, and the like, refer to the action and processes of computer systems, and/or similar consumer and/or industrial electronic devices and/or machines, that manipulate and/or transform data represented as physical (electrical and/or electronic) quantities within the computer's and/or machine's registers and memories into other data similarly represented as physical quantities within the machine and/or computer system memories or registers or other such information storage, transmission and/or display devices.
• What has been described above includes examples of aspects of the disclosed subject matter. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the disclosed subject matter, but one of ordinary skill in the art may recognize that many further combinations and permutations of the disclosed subject matter are possible. Accordingly, the disclosed subject matter is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the terms “includes,” “has,” or “having,” or variations thereof, are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.

Claims (20)

What is claimed is:

1. A system that facilitates access to one or more sleep mode selection policies, comprising:

a semi-Markov decision process (MDP) component configured to access at least one wireless system parameter and determine the one or more sleep mode selection policies based on an MDP model.

2. The system of claim 1, wherein the at least one wireless system parameter includes a traffic demand parameter.

3. The system of claim 1, wherein the at least one wireless system parameter includes at least one of a mobile subscriber station parameter, a cost metric parameter, a quality of service parameter, or an optimization target.

4. The system of claim 1 further comprising a cost metric component configured to compute at least one cost metric value based on at least one corresponding action taken within the MDP model.

5. The system of claim 4, wherein the cost metric component is further configured to compute at least one cost metric value for at least one of a sleep ratio, an energy cost, or a packet delay.

6. The system of claim 1, further comprising a policy optimizer component configured to determine one or more optimized sleep mode selection policies as a subset of a total set of sleep mode policies.

7. The system of claim 6, wherein the one or more of the optimized sleep mode selection policies satisfy a quality of service requirement.

8. The system of claim 6, wherein the policy optimizer component is configured to apply a linear programming policy optimization algorithm during a determination of the one or more optimized sleep mode selection policies.

9. The system of claim 8, wherein the linear programming policy optimization algorithm minimizes a power consumption of the system subject to a delay constraint of the system.

10. The system of claim 1, wherein the one or more sleep mode selection policies are employed to determine at least one timeout delay value or a sleep mode transition value.

11. The system of claim 10, wherein the one or more sleep mode selection policies are in the form of one or more state-action frequencies translatable into one or more command probabilities.

12. The system of claim 1, wherein the wireless system employs wireless technologies conforming to the Institute of Electrical and Electronics Engineers (IEEE) 802.16e-2005 standard.

13. A method, comprising:

accessing at least one wireless system parameter;

determining at least one optimized sleep mode selection policy; and

facilitating access to the at least one optimized sleep mode selection policy.

14. The method of claim 13, wherein the wireless system employs wireless technologies conforming to the Institute of Electrical and Electronics Engineers (IEEE) 802.16e-2005 standard.

15. The method of claim 13, wherein the accessing includes accessing at least one of an expected sleep ratio, an expected power consumption level, or an expected packet delay value.

16. The method of claim 13, further comprising generating a timeout value or transition decision value based, at least in part, on the at least one optimized sleep mode selection policy.

17. The method of claim 16, wherein the at least one optimized sleep mode selection policy is in the form of a set of state-action frequencies translatable into a set of command probabilities.

18. The method of claim 13, wherein the optimized sleep mode selection policy further satisfies a quality of service requirement.

19. A method of optimizing sleep mode operation for a mobile subscriber station (MSS) in a wireless communications system, comprising:

modeling sleep mode operation of the MSS as a semi-Markov decision process (MDP) to facilitate computing performance evaluations of wireless power saving classes associated with the MSS and the wireless communications system;

modeling at least a cost function for at least a sleep mode behavior of the MSS; and

selecting at least an optimized sleep mode selection policy from a set of sleep mode selection policies for the MSS based at least in part on minimizing power consumption of the MSS.

20. An article of manufacture including a computer-readable medium having instructions stored thereon that, in response to execution by a computing device, cause the computing device to perform a method, comprising:

modeling a sleep mode of a mobile subscriber station (MSS) as a semi-Markov decision process (MDP) to facilitate evaluations of wireless power saving classes associated with the MSS and the wireless communications system;

modeling at least a cost function of the sleep mode; and

selecting at least an optimized sleep mode selection policy from a set of sleep mode selection policies for the MSS based at least in part on minimizing power consumption of the MSS.

Images