CN109617939B - WebIDE cloud server resource allocation method based on task pre-scheduling - Google Patents

Abstract

The invention discloses a WebIDE cloud server resource allocation method based on task pre-scheduling, which comprises the following steps of: dividing WebIDE system tasks; step 2: constructing a task prediction model based on the Markov state transition probability matrix; and step 3: WebIDE task pre-scheduling based on a task prediction model; and 4, step 4: and (4) allocating resources of the cloud server. The method comprises the steps of classifying tasks according to the processing types of user tasks by a WebIDE system, analyzing the conversion relation among the tasks, and then constructing an operation task prediction model by using a Markov state transition probability matrix; designing a task prescheduler according to the prediction model, and prescheduling the task; and finally, carrying out cloud server resource allocation by combining task pre-scheduling and an ant colony algorithm. Experiments prove that compared with an ant colony algorithm, the method can effectively reduce task response time delay and improve the resource utilization rate of the cloud server.

Description

WebIDE cloud server resource allocation method based on task pre-scheduling

Technical Field

The invention belongs to the field of cloud server resource allocation, and particularly relates to design and implementation of a WebIDE cloud server resource allocation method based on task pre-scheduling.

Background

With the use of new technologies such as visual interaction and cloud server, and the rise of Web applications, people move locally executed applications to the cloud, and a cloud-based ide (webide) is also proposed and gradually becomes a hotspot of academic research. Many companies have begun to use webides for application development, and the mature webides now include Coding WebIDE, Cloud9, Eclipse che, and Codenvy, among others. The WebIDE system relies on a cloud server and a browser, provides visual file management and code editing for a user, and supports functions of one-stop deployment, code operation and the like in a programming environment. The cloud server is an application which relies on a server cluster and provides functions of fast computing, safe storage and the like for the outside. The method is developed on the basis of distributed computing, parallel computing, network storage and virtualization technologies. The cloud server is different from a traditional PC (personal computer) era computing and storing mode, takes a network as a central medium, and places data storage and computing in a remote cluster service center for processing. For users, the cluster service center is similar to an intangible "cloud" end. Cloud servers generally have four features: virtualization, super-scale, high reliability, on-demand service. When high concurrency of WebIDE tasks is faced, the reasonable resource allocation method of the cloud server can improve the resource utilization rate and reduce the response time of the WebIDE tasks.

As can be seen from the analysis of the existing research, the problem of cloud server resource allocation is basically researched on the premise that WebIDE tasks are determined at present. In actual resource allocation, if the arrival time, execution time and required resources of the WebIDE task can be predicted, the resource utilization rate can be significantly improved. However, in the actual application of the cloud server, a large number of dynamic factors exist, such as the dynamic arrival of the WebIDE task and the randomness of the execution time of the WebIDE task, and the factors affect the accuracy of predicting the arrival time, the execution time and the required resources of the WebIDE task by the cloud server. Therefore, in situations where the WebIDE task is undetermined, there are certain challenges to studying the cloud server resource allocation problem.

Disclosure of Invention

The invention aims to provide a WebIDE cloud server resource allocation method based on task pre-scheduling so as to solve the technical problem.

In order to achieve the purpose, the invention adopts the following technical scheme:

a WebIDE cloud server resource allocation method based on task pre-scheduling comprises the following steps:

step 1: dividing WebIDE system tasks;

step 2: constructing a task prediction model based on the Markov state transition probability matrix;

and step 3: WebIDE task pre-scheduling based on a task prediction model;

and 4, step 4: and (4) allocating resources of the cloud server.

As a further aspect of the present invention, step 1 specifically includes the following steps:

step 1.1: defining a WebIDE system task model; as shown in equation (1);

T＝(DC,ATs,DIs) (1)；

where DC is the function description, ATs is the set of actions taken, ATs ═ AT₁,AT₂,...,AT_m}; DIs is a set of resource items, DIs ═ DI₁,DI₂,..,DI_n}；

Step 1.2: defining a WebIDE task granularity model; as shown in equation (2);

P＝{AT,DI} (2)；

the AT is an action taken by a task, the DI is a resource item used by the task, and when the action and the resource item in the task granularity are the same, the actions and the resource item are considered to belong to the same task granularity;

instantiating an action set ATs and a resource item set DIs in a task model T according to the general functions of a WebIDE system, wherein the action set ATs and the resource item set DIs are respectively shown in a formula (3) and a formula (4);

ATs is { virtual machine and engineering creation, code download, online programming, code operation, code submission } (3);

(4) DIs ═ CPU, memory, compiled environment, engineering documents, code, network };

definition of T_aFor tasks before division, T_bIs a set of divided tasks. Then T_aPartitioning into T according to task granularity_bAs shown in equation (5);

step 1.3: determining a dependency relationship between WebIDE tasks;

according to the action set ATs instantiated by the formula (3) and the WebIDE system requirement, a front and back dependency set TY of the task action can be obtained;

constructing a dependency set R between the divided tasks according to the formula (5) and the formula (6);

step 1.4: determining a conversion model between WebIDE tasks;

analyzing the dependency set R item by item, and obtaining a task conversion model according to the running dependency relationship among tasks;

the conversion probability between tasks is described by using a two-dimensional matrix TM [5] [5], and TM [ i ] [ j ] represents the probability of converting the task i into the task j; in the transition probability matrix, a value of 0 indicates that there is no interconversion between the two tasks; the sum of each row or each column of the matrix is 1, which represents a complete system task conversion process;

different task execution indicates that the system is in different execution states;

the mapping between the task and the system execution state can be performed by equation (8), and thus the transition probability of the task corresponds to the transition probability of the system execution state.

As a further aspect of the present invention, step 2 specifically includes the following steps:

step 2.1: solving the state transition probability matrix by adopting a linear equation system method;

assuming that n mutually incompatible system states exist, the state i after k steps of transition is represented as S_kiAs shown in formula (9), S_kiThe state conversion process can be obtained by k steps of transition from any state, wherein each row of the matrix represents the state where one transition can be located;

for convenience of explanation, the row vector is represented as s (k) ═ s_k1 s_k2 ... s_kn]X, Y are each composed of k-1 row vectors;

X＝[s(1) s(2) ... s(k-1)]^T；

Y＝[s(2) s(3) ... s(k)]^T；

assuming that there is one state transition probability matrix P:

wherein, P_ijRepresenting the probability of state i transitioning to state j, state s₂₁Is multiplied by p by the state S (1)_*1Obtained, i.e. s₂₁＝S(1)·p_*1State s of_2nIs multiplied by p by the state S (1)_*nObtained, i.e. s_2n＝S(1)·p_*nFrom the above rule, the following linear equation set can be obtained;

the coefficient matrix of the linear equation set is X, the constant term vector is Y, and the above equation set is written in a matrix form XP ═ Y, and the solution is shown in equation (10):

P＝X^-1Y (10)；

p is a solved state transition probability matrix;

step 2.2: constructing a Markov WebIDE task prediction model according to the solved state transition probability matrix;

assuming that there are 5 execution states in the system, the initial state vector of the system is:

S(0)＝[S₁(0),...,S_j(0),...,S_n(0)],n＝5；

wherein S_j(0) Is the initial probability that the system is in state j;

the probability of the system being in the state j after k steps of transfer is S_j(k) And if so, the state vector after the k steps of transfer is as follows:

S(k)＝[S₁(k),...,S_j(k),...,S_n(k)],n＝5；

thus, the markov-based task prediction model is:

where p is the probability matrix solved, S (0). p may be used when the initial state S (0) is known^kSolving the system execution state when k is obtained;

combining equations (7), (8) and (10), the solved system execution state prediction model is converted into a task prediction model, and then an initial task vector is defined as:

T(0)＝[T_b1(0),...,T_bj(0),...,T_bn(0)],n＝5；

the system execution state transition probability is the system task transition probability, so the task transition prediction model solved according to the system execution state prediction model is shown as the formula (11);

T(k)＝T(k-1)·p＝T(0)·p^k (11)。

as a further aspect of the present invention, step 3 specifically includes the following steps:

step 3.1: defining a Virtual Machine (VM) task queue, and executing a task and a VM predicting task by the VM;

step 3.2: constructing a total WebIDE task queue, and accommodating all WebIDE tasks needing to be executed by a Virtual Machine (VM), including newly arrived tasks and tasks waiting for scheduling;

step 3.3: monitoring the resource use condition in the whole cloud server through a resource monitor;

step 3.4: according to the resource demand of the prediction task and the resource condition in the current cloud server, resource scheduling is carried out through a resource scheduler, and resource preparation is made for the execution of the prediction WebIDE task;

step 3.5: comparing the predicted WebIDE task with a task which belongs to the virtual machine and is to be executed in a system task queue through a task comparator;

step 3.6: and finishing the WebIDE task pre-scheduling, and taking the comparison result as the basis for adjusting the prediction model.

As a further aspect of the present invention, step 4 specifically includes the following steps:

step A: acquiring the currently executed WebIDE task, and predicting the next task by using the prediction model constructed in the step 3;

and B: calling the predicted WebIDE task into a virtual machine task queue to perform task pre-scheduling;

and C: when the predicted WebIDE task needs to be executed, the resource allocation is carried out by using an ant colony algorithm.

As a further aspect of the present invention, the specific resource allocation steps using the ant colony algorithm are as follows:

step 4.1: setting coefficients in the ant colony algorithm transfer function;

step 4.2: randomly distributing the tasks to the cloud server nodes;

step 4.3: according to a formula (12), carrying out selective transfer on the cloud server nodes;

among them, allowed_kRepresenting the kth path of all paths through which the ant can pass at point i, and eta (i, j) represents heuristic information and is generally represented by the reciprocal of the distance between point i and point j;

step 4.4: updating the pheromone concentration of the cloud server node according to a formula (13);

wherein n represents the number of ants, ρ is pheromone evaporation concentration, and is defined to be greater than 0 and smaller than 1. Delta tau_i,jThe pheromone quantity released by the kth ant on the route of point i and point j;

step 4.5: calculating the probability of the cloud server node corresponding to the task, and adding the calculation result into the next circular queue for comparison;

step 4.6: adding 1 to the cycle number, if k is less than the total number of tasks, returning to update the pheromone according to the formula (13), and selecting to restart from the step (3);

step 4.7: and finding out an optimal solution and outputting a selection result of the cloud server node.

The invention has the beneficial effects that: according to the method, the WebIDE task prediction model is constructed through Markov, resource allocation is carried out by combining an ant colony algorithm, the response time delay of the WebIDE task is reduced, and the resource utilization rate is improved.

Drawings

Fig. 1 is a diagram of a resource allocation model of a cloud server based on task pre-scheduling according to the present invention.

FIG. 2 is a diagram of a model for transition between tasks according to the present invention.

FIG. 3 is a diagram of a task pre-scheduling model according to the present invention.

Fig. 4 is a flow chart of cloud server resource allocation according to the present invention.

Detailed Description

The technical solution in the embodiments of the present invention will be clearly and completely explained below with reference to the drawings in the embodiments of the present invention.

As shown in fig. 1-4, a method for allocating resources to a WebIDE cloud server based on task pre-scheduling includes the following steps:

step 1: dividing WebIDE system tasks;

step 1.1: defining a WebIDE system task model; as shown in equation (1);

T＝(DC,ATs,DIs) (1)；

where DC is the functional description and ATs is the set of actions taken，ATs＝{AT₁,AT₂,...,AT_m}; DIs is a set of resource items, DIs ═ DI₁,DI₂,..,DI_n}；

Step 1.2: defining a WebIDE task granularity model; as shown in equation (2);

P＝{AT,DI} (2)；

the AT is an action taken by a task, the DI is a resource item used by the task, and when the action and the resource item in the task granularity are the same, the actions and the resource item are considered to belong to the same task granularity;

instantiating an action set ATs and a resource item set DIs in a task model T according to the general functions of a WebIDE system, wherein the action set ATs and the resource item set DIs are respectively shown in a formula (3) and a formula (4);

ATs is { virtual machine and engineering creation, code download, online programming, code operation, code submission } (3);

(4) DIs ═ CPU, memory, compiled environment, engineering documents, code, network };

definition of T_aFor tasks before division, T_bIs a set of divided tasks. Then T_aPartitioning into T according to task granularity_bAs shown in equation (5);

step 1.3: determining a dependency relationship between WebIDE tasks;

according to the action set ATs instantiated by the formula (3) and the WebIDE system requirement, a front and back dependency set TY of the task action can be obtained;

constructing a dependency set R between the divided tasks according to the formula (5) and the formula (6);

step 1.4: determining a conversion model between WebIDE tasks;

analyzing the dependency set R item by item, and obtaining a task conversion model according to the running dependency relationship among tasks; as shown in fig. 2.

As can be seen from FIG. 2, the transition probability between tasks is described by using a two-dimensional matrix TM [5] [5], and TM [ i ] [ j ] represents the probability that task i is transformed into task j; in the transition probability matrix, a value of 0 indicates that there is no interconversion between the two tasks; the sum of each row or each column of the matrix is 1, which represents a complete system task conversion process;

different task execution indicates that the system is in different execution states;

the mapping between the task and the system execution state can be performed by equation (8), and thus the transition probability of the task corresponds to the transition probability of the system execution state.

Step 2: constructing a task prediction model based on the Markov state transition probability matrix;

step 2.1: solving the state transition probability matrix by adopting a linear equation system method;

assuming that n mutually incompatible system states exist, the state i after k steps of transition is represented as S_kiAs shown in formula (9), S_kiThe state conversion process can be obtained by k steps of transition from any state, wherein each row of the matrix represents the state where one transition can be located;

for convenience of explanation, the row vector is represented as s (k) ═ s_k1 s_k2 ... s_kn]X, Y are each composed of k-1 row vectors;

X＝[s(1) s(2) ... s(k-1)]^T；

Y＝[s(2) s(3) ... s(k)]^T；

assuming that there is one state transition probability matrix P:

wherein, P_ijRepresenting the probability of state i transitioning to state j, state s₂₁Is multiplied by p by the state S (1)_*1Obtained, i.e. s₂₁＝S(1)·p_*1State s of_2nIs multiplied by p by the state S (1)_*nObtained, i.e. s_2n＝S(1)·p_*nFrom the above rule, the following linear equation set can be obtained;

the coefficient matrix of the linear equation set is X, the constant term vector is Y, and the above equation set is written in a matrix form XP ═ Y, and the solution is shown in equation (10):

P＝X^-1Y (10)；

p is a solved state transition probability matrix;

step 2.2: constructing a Markov WebIDE task prediction model according to the solved state transition probability matrix;

assuming that there are 5 execution states in the system, the initial state vector of the system is:

S(0)＝[S₁(0),...,S_j(0),...,S_n(0)],n＝5；

wherein S_j(0) Is the initial probability that the system is in state j;

the probability of the system being in the state j after k steps of transfer is S_j(k) And if so, the state vector after the k steps of transfer is as follows:

S(k)＝[S₁(k),...,S_j(k),...,S_n(k)],n＝5；

thus, the markov-based task prediction model is:

where p is the probability matrix solved, S (0). p may be used when the initial state S (0) is known^kSolving the system execution state when k is obtained;

combining equations (7), (8) and (10), the solved system execution state prediction model is converted into a task prediction model, and then an initial task vector is defined as:

T(0)＝[T_b1(0),...,T_bj(0),...,T_bn(0)],n＝5；

the system execution state transition probability is the system task transition probability, so the task transition prediction model solved according to the system execution state prediction model is shown as the formula (11);

T(k)＝T(k-1)·p＝T(0)·p^k (11)。

and step 3: WebIDE task pre-scheduling based on a task prediction model;

assuming that a VM task queue of a virtual machine refers to a specific virtual machine task queue, a task being executed on the virtual machine is called a "VM execution task"; the predicted next task that should be executed on the virtual machine is called a "VM predicted task"; during the execution of the "VM execute task", the next task actually triggered by the user is called "VM task 1, VM …", and is stored in the system task queue to be called for execution.

Step 3.1: and predicting the next task according to the currently executed task in the task queue. The prediction result is a 'VM prediction task' in the graph 3, and the prediction task is transferred into a virtual machine task queue to be used as a task to be executed;

and 3.2, acquiring resource data required by the execution of the historical task similar to the 'VM predicted task', and taking the resource data as the resource data required by the execution of the 'VM predicted task'. The resource scheduler adjusts resources according to resource data required by the VM prediction task and resource monitoring data of the resource monitor, and prepares resources for executing the VM prediction task;

step 3.3: comparing the 'VM prediction task' with 'VM task 1' in a system task queue by using a task comparator, if the tasks are the same, indicating that the prediction is accurate, directly calling and executing the 'VM prediction task', if the tasks are different, indicating that the prediction is wrong, abandoning the 'VM prediction task', and calling and executing the 'VM task 1' in the system task queue to a VM virtual machine task queue.

According to the task pre-scheduling process, the task prediction is carried out on the premise of ensuring the prediction accuracy, so that the next task can enter the virtual machine task queue in advance to wait for execution, and the cloud server resource scheduling can be carried out according to the predicted task in advance to prepare resources for the next task execution.

And 4, step 4: and (4) allocating resources of the cloud server.

And combining the ant colony algorithm with task pre-scheduling to realize cloud server resource allocation. The specific resource allocation flow is shown in fig. 4. The steps are as follows.

Step 4.1: setting coefficients in the ant colony algorithm transfer function;

step 4.2: randomly distributing the tasks to the cloud server nodes;

step 4.3: according to a formula (12), carrying out selective transfer on the cloud server nodes;

among them, allowed_kRepresenting the kth path of all paths through which the ant can pass at point i, and eta (i, j) represents heuristic information and is generally represented by the reciprocal of the distance between point i and point j;

step 4.4: updating the pheromone concentration of the cloud server node according to a formula (13);

wherein n represents the number of ants, ρ is pheromone evaporation concentration, and is greater than0 is less than 1. Delta tau_i,jThe pheromone quantity released by the kth ant on the route of point i and point j;

step 4.5: calculating the probability of the cloud server node corresponding to the task, and adding the calculation result into the next circular queue for comparison;

step 4.6: adding 1 to the cycle number, if k is less than the total number of tasks, returning to update the pheromone according to the formula (13), and selecting to restart from the step (3);

step 4.7: and finding out an optimal solution and outputting a selection result of the cloud server node.

The method comprises the steps of classifying tasks according to the processing types of user tasks by a WebIDE system, analyzing the conversion relation among the tasks, and then constructing an operation task prediction model by using a Markov state transition probability matrix; designing a task prescheduler according to the prediction model, and prescheduling the task; and finally, carrying out cloud server resource allocation by combining task pre-scheduling and an ant colony algorithm. Experiments prove that compared with an ant colony algorithm, the method can effectively reduce task response time delay and improve the resource utilization rate of the cloud server.

The foregoing is a preferred embodiment of the present invention, and it will be apparent to those skilled in the art that variations, modifications, substitutions and alterations can be made in the embodiment without departing from the principles and spirit of the invention.

Claims (3)

1. A WebIDE cloud server resource allocation method based on task pre-scheduling is characterized by comprising the following steps:

step 1: dividing WebIDE system tasks;

step 2: constructing a task prediction model based on the Markov state transition probability matrix;

and step 3: WebIDE task pre-scheduling based on a task prediction model;

and 4, step 4: resource allocation of a cloud server;

the step 1 specifically comprises the following steps:

step 1.1: defining a WebIDE system task model; as shown in equation (1);

T＝(DC,ATs,DIs) (1)；

where DC is the function description, ATs is the set of actions taken, ATs ═ AT₁,AT₂,...,AT_m}; DIs is a set of resource items, DIs ═ DI₁,DI₂,..,DI_n}；

Step 1.2: defining a WebIDE task granularity model; as shown in equation (2);

P＝{AT,DI} (2)；

the AT is an action taken by a task, the DI is a resource item used by the task, and when the action and the resource item in the task granularity are the same, the actions and the resource item are considered to belong to the same task granularity;

instantiating an action set ATs and a resource item set DIs in a task model T according to the general functions of a WebIDE system, wherein the action set ATs and the resource item set DIs are respectively shown in a formula (3) and a formula (4);

ATs is { virtual machine and engineering creation, code download, online programming, code operation, code submission } (3);

(4) DIs ═ CPU, memory, compiled environment, engineering documents, code, network };

definition of T_aFor tasks before division, T_bFor a set of divided tasks, then T_aPartitioning into T according to task granularity_bAs shown in equation (5);

step 1.3: determining a dependency relationship between WebIDE tasks;

according to the action set ATs instantiated by the formula (3) and the WebIDE system requirement, a front and back dependency set TY of the task action can be obtained;

constructing a dependency set R between the divided tasks according to the formula (5) and the formula (6);

step 1.4: determining a conversion model between WebIDE tasks;

analyzing the dependency set R item by item, and obtaining a task conversion model according to the running dependency relationship among tasks;

the conversion probability between tasks is described by using a two-dimensional matrix TM [5] [5], and TM [ i ] [ j ] represents the probability of converting the task i into the task j; in the transition probability matrix, a value of 0 indicates that there is no interconversion between the two tasks; the sum of each row or each column of the matrix is 1, which represents a complete system task conversion process;

different task execution indicates that the system is in different execution states;

the mapping between the task and the system execution state can be performed by equation (8), and thus the transition probability of the task corresponds to the transition probability of the system execution state.

2. The method for allocating resources to the WebIDE cloud server based on task pre-scheduling as claimed in claim 1, wherein step 3 specifically comprises the steps of:

step 3.1: defining a Virtual Machine (VM) task queue, and executing a task and a VM predicting task by the VM;

step 3.2: constructing a total WebIDE task queue, and accommodating all WebIDE tasks needing to be executed by a Virtual Machine (VM), including newly arrived tasks and tasks waiting for scheduling;

step 3.3: monitoring the resource use condition in the whole cloud server through a resource monitor;

step 3.4: according to the resource demand of the prediction task and the resource condition in the current cloud server, resource scheduling is carried out through a resource scheduler, and resource preparation is made for the execution of the prediction WebIDE task;

step 3.5: comparing the predicted WebIDE task with a task which belongs to the virtual machine and is to be executed in a system task queue through a task comparator;

step 3.6: and finishing the WebIDE task pre-scheduling, and taking the comparison result as the basis for adjusting the prediction model.

3. The method for allocating the resources of the WebIDE cloud server based on the task pre-scheduling as claimed in claim 1, wherein the step 4 specifically comprises the following steps:

step A: acquiring the currently executed WebIDE task, and predicting the next task by using the prediction model constructed in the step 3;

and B: calling the predicted WebIDE task into a virtual machine task queue to perform task pre-scheduling;

and C: when the predicted WebIDE task needs to be executed, the resource allocation is carried out by using an ant colony algorithm.

Images