CN102662764A - Dynamic cloud computing resource optimization allocation method based on semi-Markov decision process (SMDP) - Google Patents

Abstract

The invention discloses a dynamic cloud computing resource optimization allocation method based on a semi-Markov decision process (SMDP) and belongs to technical field of computer communication. The dynamic cloud computing resource optimization allocation method includes that 1) a could computing service domain system enables user satisfaction to be divided into N classes; 2) a terminal user sends service requests to a cloud computing service domain and applies for use of cloud computing service; 3) the cloud computing service domain system establishes an action set according to the received service requests and current cloud computing service domain states; 4) long-term income of the cloud computing service domain is computed aiming at each action in the action set; and 5) the cloud computing service domain system determines whether to accept the current service requests according to the computed long-term income, if the service requests are accepted, the cloud computing service domain system selects a virtual memory (VM) resource allocation plan corresponding to the action with maximum long-term income to allocate the VM for the cloud computing service requests. Compared with the prior art, the dynamic cloud computing resource optimization allocation method based on the SMDP greatly improves user satisfaction and service quality.

Description

Dynamic cloud computing resource optimal allocation method based on SMDP

Technical Field

The invention belongs to the technical field of computer communication, relates to a resource optimization allocation method of a cloud computing system, and particularly relates to an optimization allocation method of cloud computing resources of a cloud computing service domain in a mobile cloud computing system.

Background

Cloud computing is a new computing service model characterized by resource-as-you-go, performance computing (Armbrust, m., Fox, a., Griffith, r., Joseph, a., Katz, r., Konwinski, a., Lee, g., Patterson, d., Rabkin, a., Stoica, i., et al, "Above the centers: a Berkeley view of closed computing". cseedelivery, University of California, Berkeley, tech. report. ucb/EECS-2009-28(2009)) (2009). Cloud computing provides a new computing mode for cloud computing service providers and personal users, and can be broadly divided into three categories, namely, infrastructure-as-a-service (IaaS), platform-as-a-service (PaaS) and software-as-a-service (SaaS). With the development of wireless communication technology and internet technology, mobile terminals will gradually replace PCs to become the most important internet access device in the world. Since a mobile terminal (MD) has more advantages (such as mobility, flexibility and sensing capability) compared to a traditional wired terminal, combining mobile computing and cloud computing technologies is naturally a new method for building mobile applications, and is now attracting more and more attention in both academic and industrial fields. Therefore, a new research field, Mobile Cloud Computing (Mobile Cloud Computing), has come to be.

In previous research on mobile cloud computing, the main research direction focuses on uploading and downloading, remote operation, dynamic organization and the like of computing tasks. An author proposes a Mobile Cloud Computing model capable of running Mobile applications on a Mobile terminal and a Cloud end in (x.li, h.z, and y.zhang, "Deploying Mobile Computing in Cloud service" in Proceedings of the First International Conference for Cloud Computing, 2009, p.301.), so that the Mobile terminal with limited resources can upload Computing, transmission and storage tasks to the Cloud end for running. The authors configure the clonecoud Cloud resources by increasing the number of executions in (b.chunk and p.manitis, "Augmented Smartphone Applications Through the Cloud Execution," in Proceedings of useneix HotOS XII, 2009.), but do not take into account the actual operating state of the user terminal. The mobile terminal performs some preliminary studies on resource allocation of the elastic application service through the Cloud computing network in (x.zhang, j.schiffman, s.gibbs, a.kunjithapatham, and s.jeong, "secure electronic applications on mobile devices for closed computing," in Proceedings of the 2009 ACM works shop on Cloud computing security, 2009, pp.127-134.). In the literature (d.huang, x.zhang, m.kang, and j.luo, "mobile: a secure mobile client frame for private mobile computing and communication," in Proceedings of 5th IEEE International Symposium on Service-organized system Engineering, 2010.), Huang et al propose a mobile cloud computing architecture that allows a mobile terminal to run a Virtual Machine (VM-Virtual Machine) that uploads the relevant applications to the cloud. The author proposes a new method for configuring virtual machines according to different flows of different regions and Improving the utilization rate of a network by optimizing the placement positions of the virtual machines in (x.meng, v.pappas, and l.zhang, "Improving the reliability of the scale of data networks with a flexible virtual machine implementation," in IEEE INFOCOM, San Diego, CA, USA, March 2010 "). In fact, since these research studies on the architecture of mobile cloud computing have been relatively sufficient, the resource allocation of mobile cloud computing will naturally become the next main research direction.

In a mobile cloud computing network, cloud computing resources (such as CPUs, memories, storage and the like) of a system are respectively allocated by a plurality of mobile cloud computing service domains based on the distributed placement of a server population in a geographic location. Each mobile cloud computing service domain is composed of a plurality of Virtual machines (VM-Virtual machines), and each Virtual Machine (VM) is composed of minimum cloud computing resources capable of processing one cloud computing service. Although the cloud computing resources of a mobile cloud computing network are generally considered unlimited compared to mobile terminals, there is still a great need to leverage the cloud computing resources in the mobile cloud computing service domain to enable low-cost operation of the mobile cloud computing network.

Currently, cloud computing is especially trueResource-optimized allocation for mobile cloud computing has been less studied. Documents (h.liang, d.huang, and d.pen, "On ecological Mobile Computing Model," in progress of the international work shop On Mobile Computing and Cloud (Mobile in connection with Mobile case), 2010.) propose an economical Mobile Cloud Computing resource allocation Model that can obtain the maximum benefit of a Mobile Cloud Computing network by optimally allocating Mobile applications between the Cloud and the Mobile terminals given a system configuration. Documents (g.wei, a.v. costs kos, y.zheng, and n.xiong, "a gate-the electronic method of resource allocation for closed computing services," 2009.) propose a game theory-based cloud computing resource allocation model that can allocate cloud computing resources according to the requirements of a mobile terminal on user quality of service (QoS). In addition, there are some documents on how a cloud computing network optimizes and allocates cloud computing resources through virtual machines or servers of a data center. In (k.lorncz, b.r.chen, j.waterman, g.werner-Allen, and m.welsh, "Resource aware mapping in the pixie os," in SenSys' 08, Raleigh, North Carolina, USA, November 2008.), the authors propose a new cloud computing operation model that not only enables users to program while mastering cloud computing resources, but also enables a distribution mode in which cloud computing services reuse cloud computing resources in a cloud computing network. Documents (k.lorncz, b.chen, j.waterman, g.werner-Allen, and m.welsh, "a structured approach for supporting high throughput event processing applications," in DEB S' 09, Nashville, TN, USA, July2009.) have studied cloud computing resource allocation for event applications in cloud computing networks. In (G.Tesauro, N.K.Jong, R.das, and M.N.Bennani, "A hybrid discovery approach to automatic resource allocation," in Proc.of ICAC-06, Dublin, Ireland, June 2006.), the authors propose a resource allocation model based on an enhanced self-learning system to dynamically allocate servers in a cloud computing network, thereby improving the efficiency of server allocation in a cloud computing networkRevenue for a cloud computing network. In (K.Bolor, R.Chirkova, Y.Viniotis, and T.Salo, "Dynamic request allocation and decoding for context information applications, sub-object to a percent resource response time slot in a distributed group," in 2^ndIn IEEE International Conference on Cloud Computing Technology and Science, Indianapolis, Indiana, USA, November 2010), the authors propose a general scheme for allocating and planning Cloud Computing service requests, which improves the revenue of Cloud Computing service providers while obtaining the service quality specified by users.

Some solutions are also provided domestically for the optimal allocation of cloud computing resources. For example, in patent application 201110097395.8 (a management and control cloud computing network technology system), the author (scotch) proposed an invention of a management and control cloud computing network technology system; in patent application 201110138021.6 (a cloud computing resource management system and method), authors (new day, song, duhai and maqiang) proposed an invention of a cloud computing resource management system and method; in patent application 201110075410.9 (management method and system of configuration information in cloud computing operating system), authors (strong and billow) proposed an invention of a management method and system of configuration information in cloud computing operating system; in patent application 201080005003.4 (system and method for automatically managing virtual resources in a cloud computing environment), authors (s.m. ewerbork) proposed an invention for a system for managing virtual resources in a cloud computing environment; in patent application 201110222073.1 (a cloud computing management system based on virtualized resources), authors (sheng yun, raney and zhou yongfeng) proposed the invention of a cloud computing management system based on virtualized resources (C2 MS). One major advantage of mobile cloud computing networks is to allow mobile terminals to run their mobile application services in the cloud. And one cloud computing service can also be allocated with cloud computing resources of multiple VMs to enable the mobile terminal to obtain higher computing and storage capabilities. When a mobile cloud computing service domain receives a cloud computing service request sent from a mobile terminal, a system needs to analyze currently available cloud computing resources and determines whether to receive the cloud computing service request or not based on an analysis result; if the decision is reception, the system further needs to decide how many cloud computing resources (i.e. the number of VMs) are specifically allocated to the cloud computing service request of the mobile terminal. If all cloud computing resources in the mobile cloud computing service domain are already occupied, the system may reject the cloud computing service request of the mobile terminal due to the shortage of the cloud computing resources (we assume that there is no queue buffer in the mobile cloud computing). The rejection of the cloud computing service request of the mobile terminal not only brings negative influence on the user satisfaction and the service quality of the mobile terminal, but also greatly reduces the net profit of the system.

System revenue for a mobile cloud computing service domain typically increases as the number of received cloud computing service requests increases. On the other hand, as the system receives more cloud computing service requests, less cloud computing resources are allocated to each cloud computing service, thereby reducing the user satisfaction of the serving mobile terminal and the system performance of the mobile cloud computing service domain. Most of the existing cloud computing resource allocation methods only consider system income, do not consider expenses caused by occupation of cloud computing resources, and do not consider user satisfaction and quality of service (QoS) of the mobile terminal. Therefore, in order to obtain the overall system benefit of the mobile cloud computing service domain, when calculating the system benefit of the mobile cloud computing service domain, not only the income of the mobile cloud computing network but also the expense caused by the occupation of cloud computing resources and the user satisfaction and quality of service (QoS) of the mobile terminal need to be considered.

Disclosure of Invention

Aiming at the problem of optimal allocation of cloud computing resources of a cloud computing service domain of a mobile cloud computing network, the invention aims to provide a dynamic cloud computing resource optimization method based on SMDP. The invention provides a dynamic cloud computing resource optimal allocation model of a mobile cloud computing service domain based on a semi-Markov decision process (SMDP), an optimal allocation decision strategy of cloud computing resources of the mobile cloud computing service domain is obtained through the model, and the maximum benefit of the mobile cloud computing service domain is obtained, wherein the benefit not only considers the income brought by receiving a cloud computing service request, but also considers the expenditure brought by the cloud computing service occupying the cloud computing resources, and the user satisfaction and the service quality (QoS) of a mobile terminal. Therefore, the method plays an important role in improving the overall benefits of the mobile cloud computing system and the satisfaction degree of the mobile terminal customer on the mobile cloud computing network, and is also the practical value of the method.

The technical scheme of the invention is as follows:

an SMDP-based dynamic cloud computing resource optimization allocation method comprises the following steps:

1) the cloud computing service domain system divides the user satisfaction into N types, and the number of Virtual Machines (VM) correspondingly distributed to the user with the satisfaction type i is k_i(ii) a Wherein k is more than or equal to 1_iK is less than or equal to K, and K is the total number of VMs in the cloud computing service domain;

2) a terminal user sends a service request to a cloud computing service domain and applies for using cloud computing service;

3) the cloud computing service domain system establishes a behavior set according to the received service request and the current cloud computing service domain state;

4) calculating a long-term revenue for a cloud computing service domain for each action in the set of actions;

5) and the cloud computing service domain system determines whether to accept the current service request according to the calculated long-term income, and selects a VM resource allocation scheme corresponding to the action with the maximum long-term income to allocate VMs for the cloud computing service request if the current service request is accepted.

Further, the state s of the cloud computing service domain is represented as s ═ s<n₁，n₂，...，n_N，e>(ii) a Wherein n is_iThe user number with the satisfaction degree category of i in the cloud computing service domain is represented by eEvents within the cloud computing service domain, e ∈ { R, D₁，D₂，D_i....，D_NR is a cloud computing service request, D_iThe number of VMs occupied by the cloud computing service with satisfaction category i is completed and released.

Further, the set of actions is <math> <mrow> <mi>A</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mo>-</mo> <mn>1</mn> </mtd> <mtd> <mi>e</mi> <mo>&Element;</mo> <mo>{</mo> <msub> <mi>D</mi> <mn>1</mn> </msub> <mo>,</mo> <msub> <mi>D</mi> <mn>2</mn> </msub> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <msub> <mi>D</mi> <mi>N</mi> </msub> <mo>}</mo> </mtd> </mtr> <mtr> <mtd> <mo>{</mo> <mn>0,1</mn> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <mi>N</mi> <mo>}</mo> <mo>,</mo> </mtd> <mtd> <mi>e</mi> <mo>=</mo> <mi>R</mi> <mo>.</mo> </mtd> </mtr> </mtable> </mfenced> <mo>;</mo> </mrow> </math> Wherein, a(s) ═ 1 indicates that the cloud computing service ends running and releases the state of occupied cloud computing resources, a(s) ═ 0 indicates that the cloud computing service domain rejects the cloud computing service request, and a(s) ═ i indicates that the cloud computing service domain receives the cloud computing service request and the cloud computing resources allocated to the cloud computing service request are k_iA VM; s represents the current state of the cloud computing service domain.

Further, the profit z (s, a) for each action a is calculated using the formula z (s, a) ═ x (s, a) - τ (s, a) y (s, a); wherein x (s, a) is the total income obtained by the cloud computing service domain when the state is s and the selected action is a, and τ (s, a) represents the expected service time for transferring to the next state j when the state is s and the selected action is a; y (s, a) represents the expenditure of the cloud computing service domain when the state is s and the selected action is a.

Further, using the formula $x(s,a)=\left\{\begin{array}{cc}-1,& e=R,a=0\\ U\left(k_i\right),& e=R,a=i\end{array}\right.$ Calculating a total revenue, U (k), obtained by a cloud computing service domain_i) As a function of performance.

Further, using the formula

Calculating the expected service time of the cloud computing service domain from the current state to the next state; wherein alpha is when two decision points are consecutiveThe discount rate in between, decision point is the point in time, τ, at which any event e occurs₁Refers to the time elapsed from the current state to the state where the next event occurs.

Further, the time τ (s, a) between two decision points follows an exponential distribution, with the average rate of occurrence of events γ (s, a) ═ τ (s, a)^-1。

Further, the cloud computing service domain system utilizes a formula <math> <mrow> <mi>v</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>z</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>&eta;</mi> <munder> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>&Element;</mo> <mi>S</mi> </mrow> </munder> <mi>p</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>|</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mi>v</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Calculating a long-term yield v(s) of the cloud computing service domain for each action a in the set of actions; wherein,

p (j | S, a) is the state transition probability, j is the next state of the mobile cloud computing service domain, γ (S, a) is the average rate of event occurrence, α is the discount rate of continuous time between two decision points, a decision point refers to the time point of any event e occurrence, S is all possible states of the cloud computing service domain, and v (j) represents the overall long-term benefit obtained when the next state j occurs.

Further, the satisfaction degree of the cloud computing user is measured by adopting an efficiency function, and the user satisfaction degree is divided into N types.

Compared with the prior art, the invention has the following positive effects:

the invention is based on a semi-Markov decision process (SMDP), firstly, a new dynamic cloud computing resource optimal allocation model of the mobile cloud computing service domain is provided, and the dynamic cloud computing resource optimal allocation decision strategy obtained through the model can not only maximize the system benefit of the mobile cloud computing service domain, but also improve the utilization rate of the cloud computing resources of the mobile cloud computing service domain and the user satisfaction and service quality (QoS) of a mobile terminal. Compared with the traditional greedy algorithm for distributing network resources, the optimization strategy obtained according to the dynamic cloud computing resource optimization distribution model of the mobile cloud computing service domain is greatly improved in system benefit and performance. As can be seen from fig. 5 and 6, as the arrival rate of the cloud computing service request increases, especially when the arrival rate of the cloud computing service request exceeds 5, the performance gain of the invention is improved by at least more than 50% compared with the traditional greedy algorithm (as shown in fig. 5), and the blocking rate of the invention is reduced by at least more than 50% compared with the traditional greedy algorithm (as shown in fig. 6).

The main contributions of the present invention are reflected in the following three aspects:

1) a dynamic cloud computing resource optimization allocation decision strategy of a mobile cloud computing service domain is deduced based on a semi-Markov decision process (SMDP).

2) The model can adaptively allocate different cloud computing resources to the cloud computing service request based on currently available cloud computing resources of the mobile cloud computing service domain, improve the utilization rate of the cloud computing resources by fully utilizing the cloud computing resources of the mobile cloud computing service domain, and obtain the maximum overall benefit of the mobile cloud computing service domain.

3) The maximum system benefit of the mobile cloud computing service domain obtained by the model considers not only income brought by the mobile cloud computing service domain receiving cloud computing service requests, but also expenses brought by occupied cloud computing resources, and also user satisfaction and service quality (QoS) of the mobile terminal. Thus, the system revenue through the model is a full overall revenue.

The dynamic cloud computing resource optimization allocation model of the mobile cloud computing service domain provided by the invention not only can improve the utilization rate of cloud computing resources of the mobile cloud computing network cloud computing service domain, but also can improve the quality of service (QoS) of mobile users. In order to verify the performance of the dynamic cloud computing resource optimization allocation model of the mobile cloud computing service domain, which is provided by the invention, the dynamic cloud computing resource optimization allocation model is compared with the performance of a traditional Greedy Algorithm (Greedy Algorithm) through experiments (R.ramjee, D.Towsley, and R.Nagarajan, "On optimal call adaptation control in cellular Networks," Wireless Networks, vol.3, No.1, pp.29-41, 1997). The experimental results show that by applying the mobile cloud computing service domain dynamic cloud computing resource optimization allocation model provided by the invention, the overall system yield of a mobile cloud computing network is improved by more than 50% compared with a greedy algorithm, and the probability of rejecting a cloud computing service request is reduced by more than 50% compared with the greedy algorithm, namely, the performance and the quality of service (QoS) of the mobile cloud computing service domain dynamic cloud computing resource optimization allocation model provided by the invention are improved by more than 50% compared with the greedy algorithm.

Drawings

FIG. 1 is a service model of a mobile cloud computing network;

FIG. 2 is a flow chart of a method of the present invention;

FIG. 3 is a function of the performance of multimedia services;

fig. 4 is a state transition diagram (N ═ 2) in which a first item on the arrowhead connecting two states (e.g., a ═ 0) indicates an action taken in the current state and a second item on the arrowhead connecting two states (e.g., a ═ 0) indicates an action taken in the current state

) Indicating a transition to the next state after a corresponding action has been taken in the current stateA transition probability;

FIG. 5 is a graph comparing the performance gains of the present invention with a conventional greedy algorithm;

FIG. 6 is a graph comparing the blocking rate of the present invention with a conventional greedy algorithm.

Detailed Description

The present invention is described in further detail below with reference to the attached drawings.

1. The method comprises the following steps of (1) algorithm description of a dynamic cloud computing resource optimization allocation model of a mobile cloud computing service domain:

one of the main advantages of the mobile cloud computing network compared with the traditional Client-Server service mode is that: when the mobile terminal uploads the application service to the cloud for operation, the mobile terminal can obtain more capacity and better performance (e.g., less processing time, saving of battery power of the mobile terminal, etc.). The uploading of the elastic application task of the mobile terminal can be realized by a Weblet which connects the cloud terminal and the mobile terminal. A Weblet can use Java or a Net or Python language independent of the platform, and can also use a platform programming language. An algorithm for uploading a Weblet from a mobile terminal to a Cloud for operation is studied in (b.chun and p.manitis, "Augmented smartphone applications Through a Cloud Execution," in Proceedings of useix HotOS XII, 2009). By uploading the elastic application service to the cloud end for running through the Weblet, the computing capability, the storage capability, the network bandwidth and the like of the mobile terminal can be greatly improved. Generally, the decision of the mobile terminal to upload the task to the cloud running depends on the state of the mobile terminal itself (e.g., CPU processing capacity of the mobile terminal, battery level, network connection quality, security considerations of the mobile terminal, etc.). In the invention, when the mobile terminal determines to upload the task to the cloud terminal for operation, the mobile terminal firstly sends a service request to the cloud terminal, if the cloud terminal receives the service request of the mobile terminal, the mobile terminal then uploads the task to the cloud terminal for operation, and after the operation is finished, the cloud terminal returns an operation result to the mobile terminal.

In the invention, computing resources and communication resources (including CPUs, storage devices, memories and the like in servers, and other routing devices and communication devices and the like) in the mobile cloud computing network are uniformly managed by Virtual Machines (VMs). As shown in fig. 1, one VM is responsible for managing uploading, unloading, and processing of weblets in a mobile cloud computing network. As described above, in the dynamic cloud computing resource optimal allocation model for the mobile cloud computing service domain provided by the present invention, one VM is the minimum cloud computing resource (CPU, memory, storage, etc.) required by the mobile cloud computing service domain to process one cloud computing service, and the cloud computing resource allocated to each VM can only process one cloud computing service request at a time. Although we can consider cloud computing resources in a mobile cloud computing network to be unlimited, in a mobile cloud computing network, cloud computing resources counted in VM numbers by a specific mobile cloud computing service domain are limited. Therefore, in a mobile cloud computing service domain, if the number of arriving cloud computing service requests exceeds the number of cloud computing resources VMs available in the service domain, the subsequent cloud computing service requests will be rejected by the service domain. On the other hand, if the number of the arriving cloud computing service requests is far lower than the number of available cloud computing resources VM in the service domain, the service domain may allocate more VM numbers to each cloud computing service request to fully utilize the cloud computing resources of the service domain, so as to improve the cloud computing resource utilization rate of the mobile cloud computing service domain and the user satisfaction and service quality (QoS) of the mobile terminal.

Therefore, the dynamic cloud computing resource optimization allocation model of the mobile cloud computing service domain provided by the invention aims to make the mobile cloud computing service domain obtain the maximum overall system benefit by fully utilizing cloud computing resources, and also can improve the cloud computing resource utilization rate, the user satisfaction degree and the service quality (QoS) of the service domain.

In the invention, a mobile cloud computing network comprising only one cloud computing service domain is considered, and the cloud computing resources of the mobile cloud computing network are set to be K Virtual Machines (VMs) in total. And K is used for representing the number of VMs of the cloud computing resources allocated to one mobile terminal, wherein K is a positive integer and satisfies the condition that K is more than or equal to 1 and less than or equal to K. In addition, we can measure the satisfaction of the mobile cloud computing users by using different performance functions (J.W.Lee, R.R.Mazumdar, and N.B.Shroff, "Non-dependent optimization and rate control for multi-class services Internet," IEEE/ACM Transactions on Networking, 2005, vol.13, No.4, pp.827-840.). For example, user satisfaction of a mobile cloud computing mobile terminal may be described by a similar Sigmoidal function,

<math> <mrow> <mi>U</mi> <mrow> <mo>(</mo> <mi>r</mi> <mo>)</mo> </mrow> <mo>=</mo> <mn>1</mn> <mo>-</mo> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <mrow> <msub> <mi>&omega;</mi> <mn>2</mn> </msub> <msup> <mi>r</mi> <mn>2</mn> </msup> </mrow> <mrow> <msub> <mi>&omega;</mi> <mn>1</mn> </msub> <mo>+</mo> <mi>r</mi> </mrow> </mfrac> <mo>)</mo> </mrow> <mo>,</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>1</mn> <mo>)</mo> </mrow> </mrow> </math>

wherein U (r) represents the user satisfaction degree of the mobile cloud computing network, r is the cloud computing resource distributed to the mobile terminal by the mobile cloud computing service domain, and omega₁And ω₂Is a parameter used to adjust the u (r) waveform, the waveform of which function is shown in fig. 3.

In general, the parameter ω₁And ω₂Is determined by the mobile cloud computing service and the end user's requirements for quality of service (QoS), effectively meaningNumber selection has a significant impact on cloud computing resource allocation for a mobile cloud computing service domain. As can be seen from formula (1), in order to improve the user satisfaction of the mobile cloud computing network, it is necessary to allocate as many cloud computing resources as possible to the mobile terminal user. However, from another aspect, in order to improve the overall system benefit of the mobile cloud computing network, according to the total available cloud computing resources of the mobile cloud computing service domain and the demand of the mobile users for the cloud computing resources by using the cloud computing service, the system cannot allocate the maximum cloud computing resources to each mobile terminal user individually. In order to establish an optimization model for dynamic requirements of cloud computing services of a mobile cloud computing network, it is assumed that a process of a mobile terminal requesting to access the mobile cloud computing network and using the cloud computing services obeys Poisson distribution (Poisson) with a mean value of λ, and meanwhile, the online time of a mobile cloud computing terminal user obeys exponential distribution with a mean value of λλ represents a rate at which cloud computing requests reach the cloud computing network with a poisson distribution, and μ represents a rate at which a user who ends a service leaves the cloud computing network following an exponential distribution.

2. As shown in fig. 2, the steps of establishing the dynamic cloud computing resource optimization allocation model of the mobile cloud computing service domain are as follows:

1) setting a system state

In order to characterize an optimized distribution model of cloud computing resources in a mobile cloud computing service domain by using a semi-Markov decision process, since the satisfaction degree of a user on a mobile cloud computing network is inversely proportional to the processing time of the service request, that is, the shorter the processing time of the service request of the user is, the higher the satisfaction degree of the user on a system is, obviously, the shorter the processing time of the service requested by the user is if the cloud computing resources (that is, the number of VMs) allocated to the user are, the more the cloud computing resources (that is, the number of VMs) allocated to the user are, generally, the cloud computing resources of a plurality of VMs can be processed in parallel, so that the. Therefore, the satisfaction degree of the user on the mobile cloud computing network and the cloud meter distributed to the user can be knownComputational resources are proportional, i.e., the more cloud computing resources allocated to a user, the higher the satisfaction of the user. We classify user satisfaction of mobile cloud computing networks into N classes. Therefore, the system state of the dynamic cloud computing resource optimization allocation model of the mobile cloud computing service domain based on the semi-mahalanobis process is defined to be the number of cloud computing services which are provided for each user satisfaction degree and the set of events which occur in the mobile cloud computing service domain. We also define k_iA number of VMs of cloud computing resources allocated to a user with a user satisfaction of i, where i is 1, 2₁＜…＜k_NK is less than or equal to K. By n_iThe number of all users with the user satisfaction degree i in the mobile cloud computing service domain is represented. User with satisfaction degree i should be assigned k_iCloud computing resources of individual VMs.

In a mobile cloud computing service domain, there are a total of two types of events:

1) a new cloud computing service request, denoted by R;

2) completing the operation of a cloud computing service with user satisfaction degree i, releasing the occupied cloud computing resources, and using D_iTo indicate.

Therefore, any event e in the mobile cloud computing service domain can be used as e { R, D ∈ [ ]₁，D₂，....，D_NExpressed with S, all possible states of the system are expressed with S, so that the system state of the mobile cloud computing service domain can be expressed with the following formula:

S＝{s|s＝<n₁，n₂，...，n_N，e>}.

(2)

2) setting action sets

When a terminal user requests to access the mobile cloud computing service domain and applies for using the cloud meterWhen calculating service (e ═ R), the mobile cloud computing service domain needs to decide whether to accept the user request, and if so, how many cloud computing resources of the VM should be allocated to the user requesting the service. For simplicity, we denote that the mobile cloud computing service domain rejects a cloud computing service request by a(s) -0; denote that the mobile cloud computing service domain receives the cloud computing service request by a(s) ═ i, and the cloud computing resource allocated to the cloud computing service request is k_iAnd the VM is used for ensuring the satisfaction degree of the end user to the cloud computing service to reach i, wherein s represents the current system state. On the other hand, we use a(s) ═ 1 to denote a state where a cloud computing service ends running and occupied cloud computing resources are released (here, event e ═ D)_i) And (5) performing the following actions, namely counting the number of VMs of the currently available cloud computing resources and waiting for the occurrence of the next event. Thus, the set of actions for this model is summarized as follows:

<math> <mrow> <mi>A</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mo>-</mo> <mn>1</mn> </mtd> <mtd> <mi>e</mi> <mo>&Element;</mo> <mo>{</mo> <msub> <mi>D</mi> <mn>1</mn> </msub> <mo>,</mo> <msub> <mi>D</mi> <mn>2</mn> </msub> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <msub> <mi>D</mi> <mi>N</mi> </msub> <mo>}</mo> </mtd> </mtr> <mtr> <mtd> <mo>{</mo> <mn>0,1</mn> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <mi>N</mi> <mo>}</mo> <mo>,</mo> </mtd> <mtd> <mi>e</mi> <mo>=</mo> <mi>R</mi> <mo>.</mo> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>3</mn> <mo>)</mo> </mrow> </mrow> </math>

3) setting a profit model

Based on the system state and the corresponding action, we can estimate the gain (represented by z (s, a), where a is the action taken by the system for each event, including rejecting the cloud computing service request or allocating the cloud computing resources of i VMs for the cloud computing service request, in the state of arrival of the cloud computing service request, a can also be understood as the number of VMs allocated to each user's cloud computing resources, but in the state of departure of the mobile user after the cloud computing service is finished, a-1 represents counting the currently available cloud computing resources), which is composed of two parts, one part is the system income, the other part is the system expenditure, and can be represented by the following formula,

z(s，a)＝x(s，a)-τ(s，a)y(s，a) (4)

x (s, a) is the total revenue the system receives when the system is in state s and the selected action is a, and may be expressed as,

$x(s,a)=\left\{\begin{array}{cc}-1,& e=R,a=0\\ U\left(k_i\right),& e=R,a=i\end{array}\right.---\left(5\right)$

wherein, U (k)_i) As a function of performance, as shown in equation (1). τ (s, a) represents the expected service time for transitioning to the next system state j when the current system state is s and the selected action is a; y (s, a) represents the expenditure when the current system state is s and the selected action is a, and y (s, a) can be measured by the total number of the cloud computing resources VM occupied by the running cloud computing service, and is represented by the following formula:

<math> <mrow> <mi>y</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>n</mi> <mi>i</mi> </msub> <msub> <mi>k</mi> <mi>i</mi> </msub> <mo>.</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>6</mn> <mo>)</mo> </mrow> </mrow> </math>

the present invention determines whether to decide the request based on the long-term expected revenue from the revenue model. The long-term profit model of the invention can respectively obtain the long-term profits of receiving and refusing according to the profit model, and then the action a (receiving or refusing) with the maximum long-term profits is selected; if the current request is received, the system state is updated. While waiting for the next event, then deciding to act on that event, and re-updating the system state, thus running in cycles.

4) Solving state transition probabilities

The decision point refers to a time point when any event occurs, for example, a cloud computing service request arrives at a mobile cloud computing service domain, or a mobile terminal user who has finished using the cloud computing service leaves the cloud computing service domain and releases occupied cloud computing resources. In our system model, since the time τ (s, a) between two decision points is exponentially distributed, the average rate of occurrence of all events γ (s, a) can be expressed as,

<math> <mrow> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>&tau;</mi> <msup> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </msup> </mrow> </math>

<math> <mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mi>&lambda;</mi> <mo>+</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>n</mi> <mi>i</mi> </msub> <mi>&mu;</mi> <mo>,</mo> </mtd> <mtd> <mi>e</mi> <mo>=</mo> <mi>R</mi> <mo>,</mo> <mi>a</mi> <mo>=</mo> <mn>0</mn> </mtd> </mtr> <mtr> <mtd> </mtd> <mtd> <mi>ore</mi> <mo>=</mo> <msub> <mi>D</mi> <mi>i</mi> </msub> </mtd> </mtr> <mtr> <mtd> <mi>&lambda;</mi> <mo>+</mo> <mrow> <mo>(</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>n</mi> <mi>i</mi> </msub> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> <mi>&mu;</mi> <mo>,</mo> </mtd> <mtd> <mi>e</mi> <mo>=</mo> <mi>R</mi> <mo>,</mo> <mi>a</mi> <mo>=</mo> <mi>i</mi> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>7</mn> <mo>)</mo> </mrow> </mrow> </math>

thus, the desired discount yield z (s, a) between times τ (s, a) may be expressed as,

<math> <mrow> <mi>z</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>x</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>y</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <msubsup> <mi>E</mi> <mi>s</mi> <mi>a</mi> </msubsup> <mo>{</mo> <msubsup> <mo>&Integral;</mo> <mn>0</mn> <msub> <mi>&tau;</mi> <mn>1</mn> </msub> </msubsup> <msup> <mi>exp</mi> <mrow> <mo>-</mo> <mi>&alpha;t</mi> </mrow> </msup> <mi>dt</mi> <mo>}</mo> </mrow> </math>

<math> <mrow> <mo>=</mo> <mi>x</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>-</mo> <mi>y</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <msubsup> <mi>E</mi> <mi>s</mi> <mi>a</mi> </msubsup> <mo>{</mo> <mfrac> <mrow> <mn>1</mn> <mo>-</mo> <msup> <mi>exp</mi> <mrow> <mo>-</mo> <msub> <mi>&alpha;&tau;</mi> <mn>1</mn> </msub> </mrow> </msup> </mrow> <mi>&alpha;</mi> </mfrac> <mo>}</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>8</mn> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <mo>=</mo> <mi>x</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>-</mo> <mfrac> <mrow> <mi>y</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> <mrow> <mi>&alpha;</mi> <mo>+</mo> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> </mfrac> </mrow> </math>

where E is the average expected time to the next event occurrence, τ, when action a is taken in state s₁Refers to the time elapsed from the current state to the state where the next event occurs. x (s, a) and y (s, a) have been defined in equations (5) and (6), respectively, and since the time between two events is continuous and we are discrete at the time point of decision making, it is necessary to convert the continuous time gains to obtain the discrete time gains, so we use α to represent the discount rate in continuous time. Let p (j | s, a) denote the transition probability of the system transitioning to state j when the selected action is a at state s. From this we can deduce all the state transition probabilities. To simplify the writing, we define the following symbols,

${\hat{n}}_1=<{\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="HDA0000157323010000021.png,HDA0000157323010000031.png"\; state="\{\{state\}\}">n</figure-callout>}}_1,{\mathrm{<figure-callout\; id="2"\; label="n"\; filenames="HDA0000157323010000021.png,HDA0000157323010000031.png"\; state="\{\{state\}\}">n</figure-callout>}}_2,..,n_i,..,n_N>$

${\hat{n}}_{2,i}=<{\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="HDA0000157323010000021.png,HDA0000157323010000031.png"\; state="\{\{state\}\}">n</figure-callout>}}_1,..,n_i-1,..,n_N>$

${\hat{n}}_{3,i}=<{\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="HDA0000157323010000021.png,HDA0000157323010000031.png"\; state="\{\{state\}\}">n</figure-callout>}}_1,..,n_i+1,..,n_N>$

${\hat{n}}_{4,i,m}=<{\mathrm{<figure-callout\; id="1"\; label="n"\; filenames="HDA0000157323010000021.png,HDA0000157323010000031.png"\; state="\{\{state\}\}">n</figure-callout>}}_1,..,n_i+1,..,n_m-1,..,n_N>.---\left(9\right)$

when a new cloud computing service request arrives at the mobile cloud computing service domain, if the decision of the system is rejection, thenMeanwhile, a is 0; or when a cloud computing service with user satisfaction degree i finishes running and the mobile terminal user leaves the mobile cloud computing service domain and releases cloud computing resources, the number of users receiving the cloud computing service in the system is reduced and the available cloud computing resources are increased, so thatIn both cases, we can obtain the transition probabilities of,

<math> <mrow> <mi>p</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>|</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mfrac> <mi>&lambda;</mi> <mrow> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mtd> <mtd> <mi>j</mi> <mo>=</mo> <mo>&lt;</mo> <msub> <mover> <mi>n</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> <mo>,</mo> <mi>R</mi> <mo>></mo> </mtd> </mtr> <mtr> <mtd> <mfrac> <msub> <mi>n</mi> <mrow> <mi>i</mi> <mo>,</mo> <mi>&mu;</mi> </mrow> </msub> <mrow> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mtd> <mtd> <mi>j</mi> <mo>=</mo> <mo>&lt;</mo> <msub> <mover> <mi>n</mi> <mo>^</mo> </mover> <mrow> <mn>2</mn> <mo>,</mo> <mi>i</mi> </mrow> </msub> <mo>,</mo> <msub> <mi>D</mi> <mi>i</mi> </msub> <mo>></mo> <mo>,</mo> <msub> <mi>n</mi> <mi>i</mi> </msub> <mo>&GreaterEqual;</mo> <mn>1</mn> <mo>.</mo> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>10</mn> <mo>)</mo> </mrow> </mrow> </math>

when a new cloud computing service request arrives at the mobile cloud computing service domain, if the decision of the system at this time is that access is granted and the cloud computing resource to be allocated for the cloud computing service request is k_iVM, then at this time

While a ═ i, i ═ 1, 2., N, in which case we can get the rotationThe probability of the shift is,

<math> <mrow> <mi>p</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>|</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mfrac> <mrow> <mrow> <mo>(</mo> <msub> <mi>n</mi> <mi>i</mi> </msub> <mo>+</mo> <mn>1</mn> <mo>)</mo> </mrow> <mi>&mu;</mi> </mrow> <mrow> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mtd> <mtd> <mi>j</mi> <mo>=</mo> <mo>&lt;</mo> <msub> <mover> <mi>n</mi> <mo>^</mo> </mover> <mn>1</mn> </msub> <mo>,</mo> <msub> <mi>D</mi> <mi>i</mi> </msub> <mo>></mo> </mtd> </mtr> <mtr> <mtd> <mfrac> <mi>&lambda;</mi> <mrow> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mtd> <mtd> <mi>j</mi> <mo>=</mo> <mo>&lt;</mo> <msub> <mover> <mi>n</mi> <mo>^</mo> </mover> <mrow> <mn>3</mn> <mo>,</mo> <mi>i</mi> </mrow> </msub> <mo>,</mo> <mi>R</mi> <mo>></mo> </mtd> </mtr> <mtr> <mtd> <mfrac> <msub> <mi>n</mi> <mrow> <mi>m</mi> <mo>,</mo> <mi>&mu;</mi> </mrow> </msub> <mrow> <mi>&gamma;</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mtd> <mtd> <mi>j</mi> <mo>=</mo> <mo>&lt;</mo> <msub> <mover> <mi>n</mi> <mo>^</mo> </mover> <mrow> <mn>4</mn> <mo>,</mo> <mi>i</mi> <mo>,</mo> <mi>m</mi> </mrow> </msub> <mo>,</mo> <msub> <mi>D</mi> <mi>m</mi> </msub> <mo>></mo> <mo>,</mo> <msub> <mi>n</mi> <mi>m</mi> </msub> <mo>&GreaterEqual;</mo> <mn>1</mn> <mo>,</mo> <mi>m</mi> <mo>&NotEqual;</mo> <mi>i</mi> <mo>.</mo> </mtd> </mtr> </mtable> </mfenced> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>11</mn> <mo>)</mo> </mrow> </mrow> </math>

fig. 4 shows a state transition diagram when N is 2 based on our proposed mobile cloud computing network dynamic cloud computing resource optimization allocation decision model.

5) Solving for maximized overall long-term benefits of the system

Therefore, according to the definition of the semi-Markov decision process (SMDP), the maximum long-term discount benefit of the SMDP-based mobile cloud computing service domain dynamic cloud computing resource optimization allocation model provided by the invention can be obtained as follows,

<math> <mrow> <mi>v</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mi>max</mi> <mrow> <mi>a</mi> <mo>&Element;</mo> <mi>A</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> </mrow> </munder> <mo>{</mo> <mi>z</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>&eta;</mi> <munder> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>&Element;</mo> <mi>S</mi> </mrow> </munder> <mi>p</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>|</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mi>v</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>}</mo> <mo>-</mo> <mo>-</mo> <mo>-</mo> <mrow> <mo>(</mo> <mn>12</mn> <mo>)</mo> </mrow> </mrow> </math>

wherein

z (s, a) and p (j | s, a) have been obtained in equations (8), (10) and (11), respectively, and v (j) represents the system obtained at the next state jThe overall long-term gain is obtained.

6) Decision to find an optimization

According to the maximum system benefit obtained by the formula (12), a system decision corresponding to the maximum benefit can be easily found, the decision is an optimization decision for performing optimization allocation on the current mobile cloud computing service domain dynamic cloud computing resources, and a strategy formed by the optimization decisions for performing optimization allocation on all the mobile cloud computing service domain dynamic cloud computing resources is an optimization decision strategy for a mobile cloud computing service domain dynamic cloud computing resource optimization allocation model.

And 5) respectively obtaining corresponding long-term benefits after taking each action on the event according to the step 5), and then selecting the action corresponding to the largest long-term benefit from the long-term benefits (namely rejecting the cloud computing service request or distributing the number of VMs of the cloud computing resource for the cloud computing service request).

Claims (9)

1. An SMDP-based dynamic cloud computing resource optimization allocation method comprises the following steps:

1) the cloud computing service domain system divides the user satisfaction into N types, and the number of Virtual Machines (VM) correspondingly distributed to the user with the satisfaction type i is k_i(ii) a Wherein k is more than or equal to 1_iK is less than or equal to K, and K is the total number of VMs in the cloud computing service domain;

2) a terminal user sends a service request to a cloud computing service domain and applies for using cloud computing service;

3) the cloud computing service domain system establishes a behavior set according to the received service request and the current cloud computing service domain state;

4) calculating a long-term revenue for a cloud computing service domain for each action in the set of actions;

5) and the cloud computing service domain system determines whether to accept the current service request according to the calculated long-term income, and selects a VM resource allocation scheme corresponding to the action with the maximum long-term income to allocate VMs for the cloud computing service request if the current service request is accepted.

2. The method of claim 1, wherein the state s of a cloud computing service domain is expressed as s ═ s<n₁，n₂，...，n_N，e>(ii) a Wherein n is_iThe user number with the satisfaction degree category of i in the cloud computing service domain, e is an event in the cloud computing service domain, and e belongs to { R, D ∈ }₁，D₂，D_i....，D_NR is a cloud computing service request, D_iThe number of VMs occupied by the cloud computing service with satisfaction category i is completed and released.

3. The method of claim 2, wherein the set of actions is <math> <mrow> <mi>A</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mo>-</mo> <mn>1</mn> </mtd> <mtd> <mi>e</mi> <mo>&Element;</mo> <mo>{</mo> <msub> <mi>D</mi> <mn>1</mn> </msub> <mo>,</mo> <msub> <mi>D</mi> <mn>2</mn> </msub> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <msub> <mi>D</mi> <mi>N</mi> </msub> <mo>}</mo> </mtd> </mtr> <mtr> <mtd> <mo>{</mo> <mn>0,1</mn> <mo>,</mo> <mo>.</mo> <mo>.</mo> <mo>.</mo> <mo>,</mo> <mi>N</mi> <mo>}</mo> <mo>,</mo> </mtd> <mtd> <mi>e</mi> <mo>=</mo> <mi>R</mi> <mo>.</mo> </mtd> </mtr> </mtable> </mfenced> <mo>;</mo> </mrow> </math> Wherein, a(s) ═ 1 indicates that the cloud computing service ends running and releases the state of occupied cloud computing resources, a(s) ═ 0 indicates that the cloud computing service domain rejects the cloud computing service request, and a(s) ═ i indicates that the cloud computing service domain receives the cloud computing service request and the cloud computing resources allocated to the cloud computing service request are k_iA VM; s represents the current state of the cloud computing service domain.

4. A method according to claim 3, characterized by calculating the benefit z (s, a) for each action a using the formula z (s, a) -x (s, a) - τ (s, a) y (s, a); wherein x (s, a) is the total income obtained by the cloud computing service domain when the state is s and the selected action is a, and τ (s, a) represents the expected service time for transferring to the next state j when the state is s and the selected action is a; y (s, a) represents the expenditure of the cloud computing service domain when the state is s and the selected action is a.

5. The method of claim 4, wherein a formula is utilized $x(s,a)=\left\{\begin{array}{cc}-1,& e=R,a=0\\ U\left(k_i\right),& e=R,a=i\end{array}\right.$ Calculating a total revenue, U (k), obtained by a cloud computing service domain_i) As a function of performance.

6. The method of claim 4, wherein a formula is utilized

Calculating the expected service time of the cloud computing service domain from the current state to the next state; where α is the discount rate of the continuous time between two decision points, where a decision point is the time point when any event e occurs, and τ₁Refers to the time elapsed from the current state to the state where the next event occurs.

7. A method according to claim 6, characterized in that the time τ (s, a) between two decision points follows an exponential distribution, the average rate of occurrence of events γ (s, a) ═ τ (s, a)^-1。

8. The method of any of claims 3 to 7, wherein the cloud computing services domain system utilizes a formula <math> <mrow> <mi>v</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>z</mi> <mrow> <mo>(</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mo>+</mo> <mi>&eta;</mi> <munder> <mi>&Sigma;</mi> <mrow> <mi>j</mi> <mo>&Element;</mo> <mi>S</mi> </mrow> </munder> <mi>p</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>|</mo> <mi>s</mi> <mo>,</mo> <mi>a</mi> <mo>)</mo> </mrow> <mi>v</mi> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>,</mo> </mrow> </math> Calculating a long-term yield v(s) of the cloud computing service domain for each action a in the set of actions; wherein,

p (j | S, a) is the state transition probability, j is the next state of the mobile cloud computing service domain, γ (S, a) is the average rate of event occurrence, α is the discount rate of continuous time between two decision points, a decision point refers to the time point of any event e occurrence, S is all possible states of the cloud computing service domain, and v (j) represents the overall long-term benefit obtained when the next state j occurs.

9. The method of claim 1, wherein a performance function is used to measure the satisfaction of cloud computing users, and wherein the user satisfaction is divided into N classes.

Images