WO2021104096A1

WO2021104096A1 - Method and apparatus for task scheduling in container cloud environment, and server and storage apparatus

Info

Publication number: WO2021104096A1
Application number: PCT/CN2020/129208
Authority: WO
Inventors: 叶可江; 孙永仲; 须成忠
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2019-11-29
Filing date: 2020-11-17
Publication date: 2021-06-03
Also published as: CN111026550A

Abstract

A method and apparatus for task scheduling in a container cloud environment, and a server and a storage apparatus. The method comprises: acquiring a historical resource utilization rate of each server in a cloud data center, and predicting a predicted resource utilization rate of each server at the next moment on the basis of the historical resource utilization rate; when an application task is received, confirming resources needing to be occupied by the application task; acquiring the current resource utilization rate of each server, and combining the predicted resource utilization rate, the resources needing to be occupied and a resource threshold value of each server to perform optimization calculation, so as to obtain a deployment matrix between the application task and the server; and generating one or more groups of containers on a target server according to the deployment matrix and the application task. By predicting a future load of a container cloud server and rationally deploying an application task in conjunction with the current load condition, the overloading of the container cloud server at a future moment after task scheduling is prevented.

Description

Task scheduling method, device, server and storage device in container cloud environment

Technical field

The present invention relates to the technical field of cloud platforms, in particular to a task scheduling method, device, server and storage device in a container cloud environment.

Background technique

Compared with virtual machine technology, container technology has lower overhead and faster speed, so it is gradually replacing virtual machine technology and applied to cloud data centers. Users order services on-demand through the container cloud platform, which reduces the investment cost of infrastructure and reduces the difficulty of maintaining hardware equipment. The container cloud distributes computing tasks on a resource pool composed of a large number of servers, enabling various application systems to obtain computing resources, data resources, and storage resources as needed. Due to the heterogeneity of container cloud servers and the complexity of application tasks, how to effectively schedule application tasks through scheduling strategies and reasonably allocate computing resources has become a key issue in container clouds.

At present, the existing technology is generally based on traditional task scheduling algorithms such as first-come-first-served, weighted round-robin, Min-Min and Max-Min. There are disadvantages such as uneven node load distribution and job starvation. At the same time, the task scheduling process Multi-resource constraints such as CPU, memory, and disk capacity need to be considered, that is, when the problem is abstracted as a multi-objective optimization problem under multi-resource constraints, the solution effect of traditional algorithms is not very satisfactory. In addition, most existing technologies only determine the assignment of tasks based on the server resource utilization rate at the scheduled time. Then there may be a situation: a large number of tasks are submitted to a server with a low current resource utilization rate but will continue to grow in the future, resulting in the next The server is overloaded at all times.

Summary of the invention

The present invention provides a task scheduling method, device, server and storage device in a container cloud environment to solve the problem that the existing task scheduling method does not consider whether the server will be overloaded in the future.

To solve the above technical problems, the present invention discloses a task scheduling method in a container cloud environment, including:

Obtain the historical resource utilization of each server in the cloud data center, and predict the predicted resource utilization of each server at the next moment based on the historical resource utilization;

When an application task is received, confirm the resources required by the application task;

When scheduling application tasks, obtain the current resource utilization of each server, and combine the predicted resource utilization, the required occupied resources and the resource threshold of each server to perform optimization calculations to obtain the deployment matrix between the application tasks and the servers;

According to the deployment matrix and application tasks, one or more sets of containers are generated on the target server.

As a further improvement of the present invention, the step of obtaining the historical resource utilization rate of each server in the cloud data center and predicting the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate includes:

Pre-set time series, which is composed of multiple historical resource utilization rates;

Collect historical resource utilization rates at preset intervals;

Update the time series based on historical resource utilization;

Input the time series into the built Prophet model, and predict the predicted resource utilization of each server at the next moment.

As a further improvement of the present invention, the step of updating the time series according to the historical resource utilization rate includes:

The new historical resource utilization rate is added to the end of the time series, and the earliest historical resource utilization rate in the time series is deleted to update the time series.

As a further improvement of the present invention, it also includes constructing a Prophet model. The steps of constructing a Prophet model include:

Set the position of the change point on the time series to divide the time series into multiple segments;

Detect the changing trend of each time series;

Use changing trends to build a trend model;

Use the preset period to build a seasonal model;

Get the number of holidays included in the time series, and use the number of holidays to construct a holiday model;

Use trend models, seasonal models, and holiday models to build Prophet models.

As a further improvement of the present invention, the trend model is:

Among them, C(t) is the resource capacity of the server at time t; the vector δ represents the change in the growth rate on the _{timestamp S j,}

Represents the S-dimensional real number space, the timestamp S _j indicates that there are S change points located in the timestamp S _j , j=1,...,S, then the growth rate at time t is k+a(t) ^T δ, k is Basic growth rate, vector a(t)∈{0,1} ^s satisfies

γ represents the adjustment amount of the preset bias parameter m at all change points

The season model is:

Among them, P is the expected period of the preset time series, a _n , b _n ∈ β, β is generated by a normal distribution in the interval from 0 to σ, and N and σ are preset;

The holiday model is:

h(t)=Z(t)k;

Among them, Z(t)=[1(t∈D ₁ ),...,1(t∈D _L )] indicates whether the time stamp t is on a holiday, D _i indicates a period of time before and after the holiday, k=(k ₁ ,... ,k _L ) ^T , k _i represents the corresponding change of the predicted holiday, k is generated by a normal distribution in the interval from 0 to v, and v is preset;

The Prophet model is:

y(t)=g(t)+s(t)+h(t)+ε _t ;

Among them, ε _t is the error term and obeys normal distribution.

As a further improvement of the present invention, the current resource utilization rate of each server is obtained, and optimized calculation is performed in combination with the predicted resource utilization rate, the required occupied resources and the resource threshold of each server to obtain the deployment matrix between the application task and the server. The steps include:

Get the current resource utilization of each server;

Construct objective function and constraint function based on current resource utilization rate, predicted resource utilization rate, required occupied resources and resource threshold, and initialize the particle swarm, set the population size and the number of iterations;

Save the non-dominated particles in the particle swarm as guiding particles in the optimal solution set and final solution set, and calculate the crowding distance of each guiding particle;

Iterate, and each particle selects a guiding particle from the optimal solution set according to the crowding distance;

Perform the flight process for each particle;

Divide the particle swarm into three parts of the same size, and adopt different mutation schemes for each part. The mutation scheme includes no mutation, uniform mutation and uneven mutation;

When the current position of the particle dominates the optimal position of the particle or the two do not dominate each other, update the optimal position of the individual to the current position, and add the particle to the optimal solution set and the final solution set;

When the position of all particles is updated, update the crowded distance of the particles;

When the maximum number of iterations is reached, the deployment matrix is obtained.

As a further improvement of the present invention, the objective function is:

Res∈{CPU,Mem,Disk};

among them,

Is the estimated value of the current resource utilization,

Is the standard deviation of the estimated value of the current resource utilization, Res represents the resource type;

The first constraint function is:

Among them, e _ij indicates whether the i-th application task is deployed to run on the j-th server, if it is, then e _ij =1, if not, then e _ij =0, the first constraint function means that each batch task can only be deployed on On the same server;

The second constraint function is:

among them,

Is the current resource utilization, B _{i, Res} are the required occupied resources of the application task, T _{i, Res} are the resource thresholds of the server,

To predict the estimated value of resource utilization,

To predict the resource utilization rate, the second constraint function indicates that when the application task is deployed to the server, the current resource utilization rate and the predicted resource utilization rate are less than or equal to the resource threshold.

In order to solve the above problems, the present invention also provides a task scheduling device in a container cloud environment, including:

The obtaining module is used to obtain the historical resource utilization rate of each server in the cloud data center, and predict the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate;

Confirmation module, used to confirm the resources occupied by the application task when the application task is received;

The optimization module is used to obtain the current resource utilization of each server when scheduling application tasks, and combine the predicted resource utilization, the required occupied resources and the resource threshold of each server to perform optimization calculations to obtain the relationship between the application task and the server Deployment matrix

The deployment module is used to generate one or more sets of containers on the target server according to the deployment matrix and application tasks.

In order to solve the above problems, the present invention also provides a server. The server includes a processor and a memory coupled with the processor, wherein:

The memory stores program instructions used to implement any one of the above-mentioned task scheduling methods in a container cloud environment;

The processor is used to execute program instructions stored in the memory to balance the load of the container cloud server cluster.

In order to solve the above-mentioned problems, the present invention also provides a storage device that stores program files that can implement any one of the above-mentioned task scheduling methods in a container cloud environment.

The beneficial effects of the present invention are: the present invention derives the predicted resource utilization rate of the server in the future by collecting the historical resource utilization rate of the server, and then combines the current resource utilization rate of the server, the predicted resource utilization rate, the resource threshold of the server and the application Tasks need to take up resources for optimization calculations, so as to obtain the optimal deployment plan for deploying tasks to the server. It comprehensively considers the impact of the current load of the server and the future load on task scheduling, and models the load balancing problem as a multi-resource constraint The multi-objective optimization problem achieves the purpose of server cluster load balancing, and at the same time avoids the problem of overloading the server where the application task is deployed at some point in the future.

Description of the drawings

FIG. 1 is a schematic flowchart of a task scheduling method in a container cloud environment according to a first embodiment of the present invention;

2 is a schematic flowchart of a task scheduling method in a container cloud environment according to a second embodiment of the present invention;

3 is a schematic flowchart of a task scheduling method in a container cloud environment according to a third embodiment of the present invention;

4 is a schematic flowchart of a task scheduling method in a container cloud environment according to a fourth embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a task scheduling apparatus in a container cloud environment according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a server according to an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a storage device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

The terms "first", "second", and "third" in the present invention are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined with “first”, “second”, and “third” may explicitly or implicitly include at least one of the features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined. All directional indicators (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship between the components in a certain posture (as shown in the drawings) , Movement status, etc., if the specific posture changes, the directional indication will also change accordingly. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.

Reference to "embodiments" herein means that a specific feature, structure or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

FIG. 1 is a schematic flowchart of a task scheduling method in a container cloud environment according to a first embodiment of the present invention. It should be noted that, if there is substantially the same result, the method of the present invention is not limited to the sequence of the process shown in FIG. 1. As shown in Figure 1, the method includes steps:

Step S1: Obtain the historical resource utilization rate of each server in the cloud data center, and predict the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate.

Specifically, in step S101, the historical resource utilization rate of the server is periodically collected, and the resource utilization rate of the server at a future time is predicted based on the historical resource utilization rate to obtain the predicted resource utilization rate.

On the basis of the foregoing embodiment, in other embodiments, as shown in FIG. 2, step S1 includes the following steps:

In step S10, a time sequence is preset, and the time sequence is composed of multiple historical resource utilization rates.

Specifically, a time sequence of length L is preset.

Step S11: Collect historical resource utilization rates at predetermined intervals.

It should be noted that the forecast period is preset.

Step S12: Update the time series according to the historical resource utilization.

Specifically, after obtaining the historical resource utilization rate of the server, the time series is updated according to the historical resource utilization rate, and the time series is updated and maintained in time to avoid excessive time span and the time series is no longer representative.

Further, step S12 specifically includes: adding a new historical resource utilization rate to the end of the time series, and deleting the earliest historical resource utilization rate in the time series to update the time series.

Step S13: Input the time series into the built Prophet model, and predict the predicted resource utilization rate of each server at the next moment.

Specifically, input the updated time series into the only constructed Prophet model to obtain the predicted resource utilization of the server at the next moment.

It should be noted that the Prophet model is pre-built. As shown in Figure 3, the steps to construct a Prophet model include:

Step S20: Set the position of the change point on the time series to divide the time series into multiple segments.

Specifically, the change point is the trend change point. The position of the change point is set in the time series. The setting of the change point can be set manually or automatically.

In step S21, the change trend of each time series is detected.

In step S22, a trend model is constructed using the change trend.

Specifically, the trend model is:

Represents the S-dimensional real number space, the timestamp S _j indicates that there are S change points located in the timestamp S _j , j=1,...,S, then the growth rate at time t is k+a(t) ^T δ, k is The pre-set basic growth rate, the vector a(t)∈{0,1} ^S satisfies

In step S23, a season model is constructed using the preset period.

Specifically, the seasonal model is:

Among them, P is the expected period of the preset time series, a _n , b _n ∈ β, and β = (a ₁ ,b ₁ ,...,a _N ,b _N ) ^T , β passes through the positive interval from 0 to σ The state distribution is generated, and N and σ are preset. According to β _{, a n} , b _n can be calculated. It should be noted that the larger N is, the more complex seasonality can be fitted, but it will increase the risk of over-fitting. The larger σ is, the more obvious the seasonal effect is.

Step S24: Obtain the number of holidays included in the time series, and use the number of holidays to construct a holiday model.

Specifically, assuming that the time series contains X holidays, for the i-th holiday, D _i represents a period of time before and after the holiday, and the indicator function Z(t)=[1(t∈D ₁ ),...,1( t∈D _L )] indicates whether the timestamp t is on a holiday, and the holiday model is:

h(t)=Z(t)k;

Among them, k=(k ₁ ,...,k _L ) ^T , k _i represents the corresponding change of the predicted holiday, which is generated by the normal distribution of the interval from 0 to v, v is preset, and the larger v is, the holiday to the holiday model The greater the impact.

In step S25, a Prophet model is constructed using a trend model, a seasonal model, and a holiday model.

Specifically, the Prophet model is:

y(t)=g(t)+s(t)+h(t)+ε _t ;

Among them, ε _t is the error term and obeys normal distribution.

Step S2: When the application task is received, it is confirmed that the application task needs to occupy resources.

Specifically, when an application task is received, the required occupied resources when executing the application task is confirmed.

Step S3: When scheduling application tasks, obtain the current resource utilization rate of each server, and perform optimization calculations based on the predicted resource utilization rate, required occupied resources, and resource thresholds of each server to obtain the deployment matrix between application tasks and servers .

Specifically, after receiving a request for scheduling application tasks, optimization calculations are performed based on current resource utilization, predicted resource utilization, required occupied resources, and resource thresholds of each server, so as to obtain a deployment matrix between application tasks and servers.

Further, on the basis of the foregoing embodiment, in other embodiments, as shown in FIG. 4, step S3 specifically includes:

Step S30: Obtain the current resource utilization rate of each server.

In step S31, an objective function and a constraint function are constructed based on the current resource utilization rate, the predicted resource utilization rate, the required occupied resources and the resource threshold, and the particle swarm is initialized, and the population size and the number of iterations are set.

Specifically, parameters such as objective function, constraint function, population size, and number of iterations are set, and the particle swarm is initialized.

Suppose S _i represents the i-th server, a server for S _i, which is estimated current resource usage footprint of a desired value and its current resource usage and application tasks, predicted resource utilization is estimated prediction resources The sum of the utilization rate and the required resources of the application task.

The objective function is:

Res∈{CPU,Mem,Disk};

among them,

Is the estimated value of the current resource utilization,

Is the standard deviation of the estimated value of the current resource utilization, Res represents the resource type, and the resource type includes CPU, Mem (memory), and Disk (disk);

The first constraint function is:

The second constraint function is:

among them,

To predict the estimated value of resource utilization,

In step S32, the non-dominant particles in the particle swarm are stored as guiding particles in the optimal solution set and the final solution set, and the crowding distance of each guiding particle is calculated.

Step S33: Iteration is performed, and each particle selects a guiding particle from the optimal solution set according to the crowding distance.

In step S34, the flight process is performed for each particle.

Specifically, the position of each example represents a candidate solution of the application task assignment problem. Suppose that in a D-dimensional search space, the position of particle i is x _i = {x _i1 ,..., x _iD }, and the velocity is v _i ={v _i1 ,...,v _iD }, then at time t, P _id ,d=1,.., D is the individual optimal position of particle i, P _gd is the global optimal position of the particle swarm, during flight , The velocity and position of particle i are updated according to the following method:

v _id (t+1)=wv _id (t)+c ₁ r ₁ (P _id -x _id (t-1))+c ₂ r ₂ (P _gd -x _id (t-1));

x _id (t+1)=x _id (t)+v _id (t);

Among them, w is the inertia weight, c ₁ and c ₂ are acceleration constants, and r ₁ and r ₂ are random numbers uniformly distributed in [0,1].

Step S35: Divide the particle swarm into three parts of the same size, and use different mutation schemes for each part.

It should be noted that the mutation scheme includes no mutation, uniform mutation and uneven mutation. When performing mutation operation, the first part of the sub-particle swarm will not be mutated, and the second part of the sub-particle swarm will be uniformly mutated. The third part of the sub-particle group undergoes non-uniform mutation.

Step S36, when the current position of the particle dominates the individual optimal position of the particle or the two do not dominate each other, the individual optimal position is updated to the current position, and the particle is added to the optimal solution set and the final solution set.

In step S37, when the positions of all particles are updated, the crowded distance of the particles is updated.

Step S38, when the maximum number of iterations is reached, the deployment matrix is obtained.

In this embodiment, the OMOPSO algorithm is used to solve the optimal deployment matrix.

Step S4: Generate one or more sets of containers on the target server according to the deployment matrix and application tasks.

Further, when the target server fails to generate the container due to insufficient resources, abnormal operation, etc., the application task is recalled and an exception message is reported.

This embodiment collects the historical resource utilization of the server to derive the predicted resource utilization of the server in the future, and then combines the current resource utilization of the server, the predicted resource utilization, the resource threshold of the server, and the resources required for application tasks. Optimize the calculation to obtain the optimal deployment plan for deploying tasks to the server. It comprehensively considers the impact of the current load and future load of the server on task scheduling, and models the load balancing problem as a multi-objective optimization problem under multi-resource constraints. In order to balance the load of the server cluster, at the same time avoid the problem of overloading of the server that deploys the application task at a certain time in the future.

Fig. 5 is a schematic structural diagram of a task scheduling apparatus in a container cloud environment according to an embodiment of the present invention. As shown in FIG. 5, the device 50 includes an acquisition module 51, a confirmation module 52, an optimization module 53 and a deployment module 54.

The obtaining module 51 is configured to obtain the historical resource utilization rate of each server in the cloud data center, and predict the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate.

The confirmation module 52 is used for confirming the resource occupied by the application task when the application task is received.

The optimization module 53 is used to obtain the current resource utilization rate of each server when scheduling application tasks, and combine the predicted resource utilization rate, the required occupied resources and the resource threshold of each server to perform optimization calculations to obtain the relationship between the application task and the server Deployment matrix.

The deployment module 54 is used to generate one or more sets of containers on the target server according to the deployment matrix and application tasks.

Optionally, the obtaining module 51 obtains the historical resource utilization rate of each server in the cloud data center, and predicts the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate. The operation may be: preset time sequence, time The sequence is composed of multiple historical resource utilization rates; the historical resource utilization rate is collected at a preset interval; the time series is updated according to the historical resource utilization rate; the time series is input to the built Prophet model, and the forecast of each server at the next moment is predicted Resource utilization.

Optionally, the operation of the obtaining module 51 to update the time series according to the historical resource utilization rate may also be: adding a new historical resource utilization rate to the end of the time series, and deleting the earliest historical resource utilization rate in the time series to correct The time series are updated.

Optionally, before the acquisition module 51 inputs the time series into the constructed Prophet model, it also needs to construct the Prophet model. The operation of constructing the Prophet model may be: setting the position of the change point on the time series to divide the time series It is multi-segment; detects the change trend of each time series; uses the change trend to construct a trend model; uses a preset period to construct a seasonal model; obtains the number of holidays included in the time series, and uses the number of holidays to construct a holiday model;

Use trend models, seasonal models, and holiday models to build Prophet models.

Optionally, the trend model is:

The season model is:

The holiday model is:

h(t)=Z(t)k;

The Prophet model is:

y(t)=g(t)+s(t)+h(t)+ε _t ;

Among them, ε _t is the error term and obeys the normal distribution.

Optionally, the optimization module 53 obtains the current resource utilization rate of each server, and performs optimization calculations in combination with the predicted resource utilization rate, the required occupied resources, and the resource threshold of each server to obtain the deployment matrix between the application task and the server. The operation can be: obtain the current resource utilization of each server; construct the objective function and constraint function based on the current resource utilization, predicted resource utilization, required occupied resources and resource threshold, initialize the particle swarm, set the population size and the number of iterations ; Save the non-dominated particles in the particle swarm as the guiding particles in the optimal solution set and the final solution set, and calculate the crowding distance of each guiding particle; iterate, and each particle from the optimal solution set according to the crowding distance Choose a guide particle; perform the flight process for each particle; divide the particle swarm into three parts of the same size, and use different mutation plans for each part. The mutation plan includes no mutation, uniform mutation, and uneven mutation; When the current position of the particle dominates the optimal position of the particle or the two do not dominate each other, the individual optimal position is updated to the current position, and the particle is added to the optimal solution set and the final solution set; when the positions of all particles After the update is complete, update the crowded distance of the particles; when the maximum number of iterations is reached, the deployment matrix is obtained.

Optionally, the objective function is:

Res∈{CPU,Mem,Disk};

among them,

Is the estimated value of the current resource utilization,

The first constraint function is:

The second constraint function is:

among them,

To predict the estimated value of resource utilization,

Please refer to FIG. 6, which is a schematic structural diagram of a server according to an embodiment of the present invention. As shown in FIG. 6, the server 60 includes a processor 61 and a memory 62 coupled to the processor 61.

The memory 62 stores program instructions for implementing the task scheduling method in the container cloud environment described in any of the foregoing embodiments.

The processor 61 is configured to execute program instructions stored in the memory 62 to balance the load of the container cloud server cluster.

The processor 61 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 61 may be an integrated circuit chip with signal processing capability. The processor 61 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component . The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

Refer to FIG. 7, which is a schematic structural diagram of a storage device according to an embodiment of the present invention. The storage device in the embodiment of the present invention stores a program file 71 that can implement all the above methods. The program file 71 can be stored in the above storage device in the form of a software product, and includes a number of instructions to enable a computer device (which can A personal computer, a server, or a network device, etc.) or a processor (processor) executes all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage devices include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. , Or computer, server, mobile phone, tablet and other server equipment.

In the several embodiments provided by the present invention, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit. The above are only the embodiments of the present invention, and do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technical fields, The same reasoning is included in the scope of patent protection of the present invention.

Claims

A task scheduling method in a container cloud environment is characterized in that it includes:

Obtain the historical resource utilization rate of each server in the cloud data center, and predict the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate;

When an application task is received, confirm that the application task needs to occupy resources;

When scheduling the application task, obtain the current resource utilization rate of each server, and perform optimization calculations in combination with the predicted resource utilization rate, the required occupied resources and the resource threshold of each server to obtain the application task and the server Deployment matrix between;

One or more sets of containers are generated on the target server according to the deployment matrix and the application tasks.
The method for task scheduling in a container cloud environment according to claim 1, wherein said obtaining the historical resource utilization rate of each server in the cloud data center, and predicting each server at the next moment based on the historical resource utilization rate The steps for predicting resource utilization include:

Pre-set a time sequence, the time sequence is composed of a plurality of historical resource utilization rates;

Collecting the historical resource utilization rate at intervals of a preset period;

Update the time series according to the historical resource utilization;

The time sequence is input to the built Prophet model, and the predicted resource utilization rate of each server at the next moment is predicted.
The task scheduling method in a container cloud environment according to claim 2, wherein the step of updating the time sequence according to the historical resource utilization rate comprises:

The new historical resource utilization rate is added to the end of the time series, and the earliest historical resource utilization rate in the time series is deleted to update the time series.
The task scheduling method in a container cloud environment according to claim 2, further comprising constructing the Prophet model, and the step of constructing the Prophet model comprises:

Setting a position of a change point on the time series to divide the time series into multiple segments;

Detect the changing trend of each time series;

Construct a trend model using the change trend;

Constructing a seasonal model using the preset period;

Acquiring the number of holidays included in the time sequence, and constructing a holiday model by using the number of holidays;

The Prophet model is constructed using the trend model, the season model, and the holiday model.
The task scheduling method in a container cloud environment according to claim 3, wherein the trend model is:

Wherein, the C(t) is the resource capacity of the server at time t; the vector δ represents the amount of change in the growth rate on the timestamp S j,

Represents the S-dimensional real number space, the timestamp S j indicates that there are S change points located in the timestamp S j , j=1,...,S, then the growth rate at time t is k+a(t) T δ, k is Basic growth rate, vector a(t)∈{0,1} S satisfies
γ represents the adjustment amount of the preset bias parameter m at all change points

The seasonal model is:

Where, P is the preset expected period of the time series, a n , b n ∈ β, β is generated by a normal distribution in the interval from 0 to σ, and N and σ are preset;

The holiday model is:

h(t)=Z(t)k;

Among them, Z(t)=[1(t∈D 1 ),...,1(t∈D L )] indicates whether the time stamp t is on a holiday, D i indicates a period of time before and after the holiday, k=(k 1 ,... ,k L ) T , k i represents the corresponding change of the predicted holiday, k is generated by a normal distribution in the interval from 0 to v, and v is preset;

The Prophet model is:

y(t)=g(t)+s(t)+h(t)+ε t ;

Among them, ε t is the error term and obeys normal distribution.
The task scheduling method in a container cloud environment according to claim 1, wherein said obtaining the current resource utilization rate of each server is combined with the predicted resource utilization rate, the required occupied resources and each server. The step of performing optimization calculation on the resource threshold of the server to obtain the deployment matrix between the application task and the server includes:

Get the current resource utilization of each server;

Constructing an objective function and a constraint function based on the current resource utilization rate, the predicted resource utilization rate, the required occupied resources, and the resource threshold, and initializing the particle swarm, setting the population size and the number of iterations;

Saving the non-dominated particles in the particle swarm as guiding particles in the optimal solution set and the final solution set, and calculating the crowding distance of each guiding particle;

Iterate, and each particle selects a guiding particle from the optimal solution set according to the crowded distance;

Perform a flight process for each of the particles;

Divide the particle swarm into three parts of the same size, and adopt different mutation schemes for each part, and the mutation scheme includes no mutation, uniform mutation, and non-uniform mutation;

When the current position of the particle dominates the individual optimal position of the particle or the two do not dominate each other, the individual optimal position is updated to the current position, and the particle is added to the optimal solution set and In the final solution set;

After the positions of all the particles have been updated, the crowded distance of the particles is updated;

When the maximum number of iterations is reached, the deployment matrix is obtained.
The task scheduling method in a container cloud environment according to claim 6, wherein the objective function is:

among them,
Is the estimated value of the current resource utilization,
Is the standard deviation of the estimated value of the current resource utilization rate, and Res represents the resource type;

The first constraint function is:

Among them, e ij indicates whether the i-th application task is deployed to run on the j-th server, if it is, then e ij =1, if not, then e ij =0, the first constraint function indicates that each batch task can only Deploy on the same server;

The second constraint function is:

among them,
Is the current resource utilization, B i, Res are the required occupied resources of the application task, T i, Res are the resource thresholds of the server,
To predict the estimated value of resource utilization,
To predict the resource utilization rate, the second constraint function indicates that when the application task is deployed to the server, the current resource utilization rate and the predicted resource utilization rate are less than or equal to the resource threshold.
A task scheduling device in a container cloud environment is characterized by comprising:

The obtaining module is used to obtain the historical resource utilization rate of each server in the cloud data center, and predict the predicted resource utilization rate of each server at the next moment based on the historical resource utilization rate;

The confirmation module is used for confirming the resource occupied by the application task when the application task is received;

The optimization module is used to obtain the current resource utilization rate of each server when scheduling the application task, and perform optimization calculations in combination with the predicted resource utilization rate, the required occupied resources and the resource threshold of each server to obtain all Describe the deployment matrix between application tasks and servers;

The deployment module is used to generate one or more sets of containers on the target server according to the deployment matrix and the application tasks.
A server, characterized in that the server includes a processor and a memory coupled with the processor, wherein:

The memory stores program instructions for implementing the task scheduling method in a container cloud environment according to any one of claims 1-8;

The processor is configured to execute the program instructions stored in the memory to balance the load of the container cloud server cluster.
A storage device, characterized in that it stores a program file capable of implementing the task scheduling method in a container cloud environment according to any one of claims 1-8.