CN115220872A

CN115220872A - Container adjusting method and device and computer readable storage medium

Info

Publication number: CN115220872A
Application number: CN202210892379.6A
Authority: CN
Inventors: 张静; 张宪波
Original assignee: Jingdong Technology Information Technology Co Ltd
Current assignee: Jingdong Technology Information Technology Co Ltd
Priority date: 2022-07-27
Filing date: 2022-07-27
Publication date: 2022-10-21

Abstract

The embodiment of the invention provides a container adjusting method and device and a computer readable storage medium, wherein the method comprises the following steps: acquiring a first resource utilization rate corresponding to each of a plurality of containers; determining a first target container with a first resource utilization rate smaller than a first preset threshold value from a plurality of containers; acquiring a target application corresponding to the first target container; acquiring a second resource utilization rate corresponding to each target application; screening through a second resource utilization rate and a preset container setting rule to determine a second target container; acquiring first historical resource data of a target application; predicting the actual container number through a preset prediction model according to the first historical resource data; and carrying out capacity reduction on the second target container according to the actual container number and the acquired real-time container number. In the scheme, the first historical resource data can be predicted through the preset prediction model to obtain the actual number of containers, so that the capacity is reduced, and the utilization rate of the container resources is improved.

Description

Container adjusting method and device and computer readable storage medium

Technical Field

The present invention relates to the field of data applications, and in particular, to a method and an apparatus for adjusting a container, and a computer-readable storage medium.

Background

With the rapid development of services, in order to meet the pressure of peak services, technical indexes such as concurrency and complexity are continuously increased, and the most direct and effective method is to invest more resources, that is, the number of containers is correspondingly and rapidly increased. The online K8S management platform solves the problem of application containerization. However, the scheduling mode of the container is single, and the screening scheduling can be performed only according to the fact that the remaining resources of the physical machine are combined with the default strategy of the native scheduler, the kube-scheduler is expanded and configured, and whether the scheduling strategy meets the requirements or not based on real-time resource use scheduling and sla scheduling, so that the allocation of the container resources is not reasonable and efficient enough, and the utilization rate of the container resources is low.

Disclosure of Invention

The embodiment of the invention provides a container adjusting method and device and a computer readable storage medium, which can improve the utilization rate of container resources.

The technical scheme of the invention is realized as follows:

the embodiment of the invention provides a container adjusting method, which comprises the following steps: acquiring a first resource utilization rate corresponding to each of a plurality of containers; wherein the plurality of containers is present in at least one machine room; a container of the plurality of containers is for a virtual resource running on a server; determining a first target container with the first resource utilization rate smaller than a first preset threshold value from the plurality of containers; acquiring a target application corresponding to the first target container; acquiring a second resource utilization rate corresponding to each target application; screening according to the second resource utilization rate and a preset container setting rule to determine a second target container; the preset container setting rule represents that the number of containers in which the same application is located at least reaches a preset number in a machine room; acquiring first historical resource data of the target application; predicting the actual container number through a preset prediction model according to the first historical resource data; the preset prediction model representation is used for predicting future resource data according to historical resource data; and carrying out capacity reduction on the second target container according to the actual container number and the acquired real-time container number.

The embodiment of the invention provides a container adjusting device which comprises an obtaining unit, a determining unit, a predicting unit and a capacity reducing unit; wherein the content of the first and second substances,

the acquiring unit is used for acquiring the first resource utilization rates corresponding to the containers respectively; wherein the plurality of containers is present in at least one machine room; a container of the plurality of containers is for a virtual resource running on a server; determining a first target container with the first resource utilization rate smaller than a first preset threshold value from the plurality of containers; acquiring a target application corresponding to the first target container;

the determining unit is used for acquiring a second resource utilization rate corresponding to each target application; screening according to the second resource utilization rate and a preset container setting rule to determine a second target container; the preset container setting rule represents that the number of containers in which the same application is located at least reaches a preset number in a machine room;

the prediction unit is used for acquiring first historical resource data of the target application; predicting the actual container number through a preset prediction model according to the first historical resource data; the preset prediction model representation is used for predicting future resource data according to historical resource data;

and the capacity reduction unit is used for reducing the capacity of the second target container according to the actual container number and the acquired real-time container number.

An embodiment of the present invention provides a container adjustment apparatus, including:

a memory for storing executable instructions;

a processor for executing executable instructions stored in the memory, the processor performing the container adjustment method when the executable instructions are executed.

The embodiment of the invention provides a computer-readable storage medium, which is characterized by storing executable instructions, and when the executable instructions are executed by one or more processors, the processors execute the container adjusting method.

The embodiment of the invention provides a container adjusting method and device and a computer readable storage medium, wherein the method comprises the following steps: acquiring a first resource utilization rate corresponding to each of a plurality of containers; wherein the plurality of containers is present in at least one machine room; a container of the plurality of containers is for a virtual resource running on a server; determining a first target container with the first resource utilization rate smaller than a first preset threshold value from the plurality of containers; acquiring a target application corresponding to the first target container; acquiring a second resource utilization rate corresponding to each target application; screening according to the second resource utilization rate and a preset container setting rule to determine a second target container; the preset container setting rule represents that the number of containers in which the same application is located at least reaches a preset number in a machine room; acquiring first historical resource data of the target application; predicting the actual container number through a preset prediction model according to the first historical resource data; the preset prediction model representation is used for predicting future resource data according to historical resource data; and carrying out capacity reduction on the second target container according to the actual container number and the acquired real-time container number. In the above scheme, the server may obtain respective first resource utilization rates corresponding to the multiple containers, and determine the first target container according to the first resource utilization rates; a container range to be shrunk can be obtained; according to a second resource utilization rate corresponding to the target application corresponding to the first target container; screening through a second resource utilization rate and a preset container setting rule to determine a second target container; the range of the container to be shrunk can be reduced, and the accuracy and efficiency of shrinking the container are improved; according to the first historical resource data of the target application, the actual container number is predicted through a preset prediction model, and the second target container is reduced according to the actual container number and the acquired real-time container number, so that the containers with low resource utilization rate can be reduced, and the utilization rate of the whole container resources is improved.

Drawings

FIG. 1 is a first schematic flow chart of an alternative method for adjusting a container according to an embodiment of the present invention;

FIG. 2 is a second schematic flow chart diagram illustrating an alternative method for adjusting a container according to an embodiment of the present invention;

FIG. 3 is a diagram of an alternative clustering result of a container adjustment method according to an embodiment of the present invention;

FIG. 4 is a third alternative flow chart of a container adjustment method according to an embodiment of the present invention;

FIG. 5 is a fourth alternative flow chart of a container adjustment method according to an embodiment of the present invention;

FIG. 6 is a fifth alternative flow chart of a container adjustment method according to an embodiment of the present invention;

FIG. 7 is a sixth alternative flow chart of a container adjustment method according to an embodiment of the present invention;

FIG. 8 is a first schematic structural view of a container adjustment apparatus according to an embodiment of the present invention;

fig. 9 is a second schematic structural diagram of a container adjustment apparatus according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.

In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. Fig. 1 is a schematic flow chart illustrating an alternative method for adjusting a container according to an embodiment of the present invention, which will be described with reference to the steps shown in fig. 1.

S101, obtaining first resource utilization rates corresponding to the containers.

In some embodiments of the invention, a plurality of containers is present in at least one machine room; a container of the plurality of containers is for a virtual resource running on a server.

In some embodiments of the present invention, the server may obtain second historical resource data corresponding to each of the plurality of containers; and determining the first resource utilization rate corresponding to each of the plurality of containers by clustering the second historical resource data.

In some embodiments of the present invention, fig. 2 is an optional schematic flow chart of a container adjustment method provided in the embodiments of the present invention, and as shown in fig. 2, S101 may be implemented through S1011 and S1012 as follows:

s1011, second historical resource data corresponding to the containers are obtained.

In some embodiments of the invention, the second historical resource data comprises a second historical processor usage and a second historical memory usage.

And S1012, clustering the second historical resource data to determine the first resource utilization rate corresponding to each of the containers.

In some embodiments of the present invention, the server may determine a cluster center of the container by clustering containers corresponding to the second historical processor usage rate and the second historical memory usage rate; and determining the first resource utilization rates corresponding to the multiple containers according to the distribution conditions of the cluster center, the second historical processor utilization rate and the second historical memory utilization rate.

It can be understood that the server may obtain second historical resource data corresponding to each of the plurality of containers; determining the respective corresponding first resource utilization rates of the plurality of containers by clustering the second historical resource data; the first resource utilization rate corresponding to each of the containers is determined, so that the resource utilization rate of each container can be analyzed, and the containers can be conveniently screened subsequently.

In some embodiments of the present invention, S1012 may be implemented by S10121 and S10122 as follows:

s10121, clustering containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate, and determining the cluster center of the containers.

In some embodiments of the present invention, the server may perform clustering on the containers by using a clustering algorithm according to the second historical processor usage rate and the second historical memory usage rate corresponding to each container, so as to determine the cluster center of the container.

It should be noted that the clustering algorithm may adopt a K-means clustering algorithm, and the embodiment of the present invention is not limited thereto.

S10122, determining the first resource utilization rates corresponding to the multiple containers according to the distribution conditions of the cluster center, the second historical processor utilization rate and the second historical memory utilization rate.

In some embodiments of the present invention, the server may compare and analyze the second historical processor usage rate and the second historical memory usage rate corresponding to the containers with a preset threshold respectively according to the distribution conditions of the cluster class center, the second historical processor usage rate, and the second historical memory usage rate, and determine the first resource usage rates corresponding to the multiple containers. Here, the preset threshold value is 15.

Illustratively, a second historical processor usage rate (corresponding to CPU usage rate), a second historical memory usage rate (corresponding to memory usage rate). In the early stage of clustering, there may be four cases according to the classification of feature classes, namely: the CPU utilization rate is high, and the memory utilization rate is high; the CPU utilization rate is low, and the memory utilization rate is high; the CPU utilization rate is high, and the memory utilization rate is low; the CPU utilization rate is low, and the memory utilization rate is low. Therefore, in the clustering process, the K-means algorithm is adopted to set the category number K =4, and the Euler distance is adopted as the measurement to cluster the use condition of the container resource, so that the result shown in FIG. 3 is finally obtained; as can be seen from the clustering result of fig. 3, the CPU utilization of the part a is less than 15, and the memory utilization is less than 15; and (3) comprehensive analysis results: part a is a case where the CPU utilization is low and the memory utilization is low. The CPU utilization rate of the part B and the part C is less than 15, and the memory utilization rate is more than 15; and (3) comprehensive analysis results: the parts B and C are the conditions of low CPU utilization rate and high memory utilization rate. The CPU utilization rate of the part D is more than 15, and the memory utilization rate is more than 15; and (3) comprehensive analysis results: the part D is the condition that the CPU utilization rate is high and the memory utilization rate is high. According to the condition that the cluster center of the cluster is combined with the resource use, the class A is marked as abnormal CPU and memory use conditions, namely the first resource use rate of the class A is low, and the class B is marked as abnormal CPU, namely the first resource use rate of the class B is low; the CD class can be considered as normal use (since the memory usage of the part C is much larger than that of the part B, the part C is judged as a storage-type container), and is marked as normal, i.e. the first resource usage of the CD class is high. And outputting the value of the center coordinate of each cluster, mapping the mapping relation between the center coordinate of each cluster and the four labels, fixing the clusters into a classification problem, classifying the containers on the real-time line, and outputting the classification labels. And meanwhile, scoring the containers according to the utilization rate of the first resources of the containers to obtain the score of the containers. The container score is transmitted to a k-means clustering algorithm from all application groups on the container according to CPU utilization and memory utilization to obtain categories of (0,0), (0,1), (1,0) and (1,1), and the category is transmitted to an isolated forest algorithm according to the category to convert the abnormal resource utilization part of each application into a score of 0-100, so that the highest application resource utilization score belonging to the category (0,0) and the lowest application resource utilization score belonging to the category (1,1) are ensured.

It can be understood that the server clusters the containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate, and determines the cluster center of the containers; determining the first resource utilization rates corresponding to the multiple containers according to the cluster center, the second historical processor utilization rate and the second historical memory utilization rate; the objectivity and accuracy of the first resource utilization rate can be guaranteed.

S102, determining a first target container with a first resource utilization rate smaller than a first preset threshold from a plurality of containers; and acquiring the target application corresponding to the first target container.

In some embodiments of the present invention, the server may screen out, according to the first resource usage rates corresponding to the multiple containers, a first target resource usage rate of which the first resource usage rate is smaller than a first preset threshold; determining a first target container according to the first target resource utilization rate; and acquiring the target application corresponding to the first target container according to the first target container.

Illustratively, from the four-type ABCD container, the a-type is marked as abnormal CPU & memory usage, i.e., the first resource usage of the a-type is low; the class B is marked as CPU abnormity, namely the first resource utilization rate of the class B is low; the CD class can be considered as normal usage and is labeled normal, i.e., the first resource usage of the CD class is high; the method comprises the steps that class AB is used as a first target container because the first resource utilization rate of the class AB is low; and acquiring the application corresponding to the first table container.

It should be noted that, a plurality of applications are arranged on the container, so that the container and the applications have an association relationship; the first target container comprises a plurality of containers of which the first resource utilization rate is smaller than a first preset threshold value.

S103, acquiring a second resource utilization rate corresponding to each target application; and screening through the second resource utilization rate and a preset container setting rule to determine a second target container.

In some embodiments of the invention, the predetermined container setting rules indicate that at least a predetermined number of containers of the same application are in a room. Presetting a container setting rule: because the containers are deployed in the machine rooms, each machine room needs to ensure at least n containers, and the minimum resource allocation of the machine rooms needs to be ensured during capacity reduction.

In some embodiments of the present invention, the server may obtain a third processor utilization rate, a third memory utilization rate, and a timing fluctuation characteristic corresponding to each target application; and calculating a second resource utilization rate corresponding to each target application according to the third processor utilization rate, the third memory utilization rate and the time sequence fluctuation characteristics. Determining a first application with a second resource utilization rate larger than a second preset threshold value from a plurality of target applications; removing containers corresponding to the first application, the newly online application and the periodically changed application from the first target container, and determining a second initial target container; acquiring machine room information corresponding to the second initial target container; and screening the second initial target container through a preset container setting rule and machine room information to determine a second target container.

In some embodiments of the present invention, fig. 4 is a schematic flow chart illustrating a third optional process of providing a container adjustment method according to an embodiment of the present invention, as shown in fig. 4, the obtaining of the second resource usage corresponding to each target application in S103 may be implemented through S201 and S202, as follows:

s201, obtaining a third processor utilization rate, a third memory utilization rate and a time sequence fluctuation characteristic corresponding to each target application.

In some embodiments of the present invention, the server may obtain a third processor usage rate, a third memory usage rate, and a timing fluctuation characteristic corresponding to each target application corresponding to each first target container.

S202, calculating a second resource utilization rate corresponding to each target application according to the third processor utilization rate, the third memory utilization rate and the time sequence fluctuation characteristics.

In some embodiments of the present invention, the server may calculate a second resource usage rate corresponding to each target application according to a third processor usage rate, a third memory usage rate, and a timing fluctuation feature corresponding to each target application, so as to obtain the second resource usage rate corresponding to each target application. And meanwhile, determining an application score according to the second resource utilization rate corresponding to each target application.

As can be understood, the server obtains the third processor utilization rate, the third memory utilization rate and the timing fluctuation feature corresponding to each target application; calculating a second resource utilization rate corresponding to each target application according to the third processor utilization rate, the third memory utilization rate and the time sequence fluctuation characteristics; the second resource utilization rate is calculated through the third processor utilization rate and the third memory utilization rate, and the objectivity and the accuracy of the second resource utilization rate can be guaranteed.

In some embodiments of the present invention, fig. 5 is a schematic view of an optional flowchart of a container adjustment method provided in an embodiment of the present invention, as shown in fig. 5, in S103, screening is performed through a second resource usage rate and a preset container setting rule, and determining a second target container may be implemented through S301 to S304, as follows:

s301, determining a first application with a second resource utilization rate larger than a second preset threshold from a plurality of target applications.

In some embodiments of the invention, the first application characterizes a normally functioning application.

In some embodiments of the present invention, the server may screen out a second target resource usage rate greater than a second preset threshold from second resource usage rates corresponding to the plurality of target applications; and determining the first application corresponding to the second target resource utilization rate according to the second target resource utilization rate.

S302, based on the first target container, removing the containers corresponding to the first application, the newly online application and the periodically changed application, and determining a second initial target container.

In some embodiments of the present invention, a newly online application refers to an application that has no historical data for one week, for a plurality of target applications, compared to the usage rate of one week; the application with the periodic variation is an application which judges whether the usage rate of the application is lower than that of the application in the same day or not according to the usage rate of a plurality of target applications in a ring mode, and if the usage rate of the application is periodically changed, the application is periodically changed.

In some embodiments of the invention, the server may determine their respective containers according to the first application, the newly online application, and the periodically changing application; and removing containers corresponding to the first application, the newly online application and the periodically changed application from the first target container to obtain a second initial target container.

It should be noted that the second initial target container includes a plurality of containers.

And S303, acquiring the machine room information corresponding to the second initial target container.

In some embodiments of the present invention, the server may determine, according to the second initial target container, information of the machine room where the second initial target container is located.

S304, screening the second initial target container through a preset container setting rule and the machine room information, and determining a second target container.

In some embodiments of the present invention, the server may divide the second initial target container according to the machine room information to obtain a plurality of container groups; for the plurality of container groups, respectively determining the fixed containers corresponding to the plurality of container groups through preset container setting rules; and removing the fixed containers from the second initial target container to determine a second target container.

It can be understood that, the server may use a target application with a high second resource usage rate as the first application, and may ensure that the remaining target applications are all with a low second resource usage rate; the method comprises the steps that containers corresponding to a first application, a newly online application and a periodically changed application can be removed from a first target container to obtain a second initial target container; the range of the container to be shrunk is reduced, and the accuracy of the container shrinkage is improved; acquiring machine room information corresponding to the second initial target container; screening the second initial target container through a preset container setting rule and machine room information to determine a second target container; the range of the container to be shrunk is further reduced, and the accuracy of shrinking the container is further improved.

In some embodiments of the present invention, S304 may be implemented by S3041, S3042, and S3043 as follows:

s3041, dividing the second initial target container according to the machine room information to obtain a plurality of container groups.

In some embodiments of the invention, the group of containers characterizes containers in one room.

In some embodiments of the present invention, the server may divide the second initial target container according to the machine room information corresponding to the second initial target container, and divide the second initial target container located in the same machine room into one container group, so as to obtain multiple container groups.

S3042, for the multiple container groups, determining the fixed containers corresponding to the multiple container groups respectively according to preset container setting rules.

In some embodiments of the present invention, the server may set rules for a plurality of container groups respectively by using preset containers, and reserve fixed containers meeting a preset number in one machine room (i.e., one container group), so as to obtain fixed containers corresponding to the plurality of container groups.

It should be noted that the fixed container is a container that must be set to ensure the normal operation of the machine room.

S3043, based on the second initial target container, removing the fixed container and determining a second target container.

In some embodiments of the present invention, the server may cull the fixed containers based on the second initial target container to obtain a second target container.

As can be understood, the server divides the second initial target container according to the machine room information to obtain a plurality of container groups; for the plurality of container groups, respectively determining the fixed containers corresponding to the plurality of container groups through preset container setting rules; based on the second initial target container, removing the fixed container, and determining a second target container; the range of the container to be shrunk is further reduced, and the accuracy of shrinking the container is further improved.

S104, acquiring first historical resource data of the target application; and predicting the actual container number through a preset prediction model according to the first historical resource data.

In some embodiments of the invention, the pre-set prediction model characterization is used to predict future resource data from historical resource data; the first historical resource data includes: first month historical resource data and first week historical resource data; the preset prediction model comprises a first prediction model and a second prediction model.

In some embodiments of the present invention, the server may perform preprocessing on the first historical resource data to obtain feature data; and judging the characteristic data, and if the characteristic data belongs to daily characteristic data, inputting the characteristic data into a first prediction model for prediction to determine the actual container number. Or if the characteristic data belongs to the large-scale characteristic data, inputting the characteristic data into the second prediction model for prediction, and determining the actual container number.

Illustratively, the predictive model employs a neural network MQ-RNN, which includes two MLP branches. First is that the global MLP will output the encoderOutbound and all future inputs are aggregated into two contexts: a series of horizon-specific contexts c _t+k Horizon agnostic context c for each of k future points, and capturing common information _a (ii) a The second local MLP, which combines the two contexts of the corresponding future input and global MLP, then outputs all quantiles needed for that particular future time step: local MLPs are used to generate sharp predictions as well as future seasonality, and adding such structure to the model can improve the stability of learning and smoothness of generating predictions.

In some embodiments of the present invention, S104 may be implemented by S1041, S1042 and S1043 as follows:

s1041, respectively preprocessing the first-month historical resource data and the first-week historical resource data to obtain month characteristic data and week characteristic data.

S1042, inputting the month characteristic data into a preset prediction model for prediction to obtain a first prediction result.

In some embodiments of the invention, the first prediction characterizes a kernel value of the processor to which the lunar data corresponds.

In some embodiments of the invention, the server may judge the month characteristic data, and if the month characteristic data belongs to the daily characteristic data, the month characteristic data is input into the first prediction model for prediction to obtain a first prediction result; or if the month characteristic data belong to the large promotion characteristic data, inputting the month characteristic data into the second prediction model for prediction to obtain a first prediction result.

And S1043, inputting the week characteristic data into a preset prediction model for prediction to obtain a second prediction result.

In some embodiments of the invention, the second prediction characterizes a kernel value of the processor to which the cycle data corresponds.

In some embodiments of the invention, the server may judge the week characteristic data, and if the week characteristic data belongs to the daily characteristic data, the week characteristic data is input into the first prediction model for prediction to obtain a second prediction result; or if the week characteristic data belongs to the promotion characteristic data, inputting the week characteristic data into the second prediction model for prediction to obtain a second prediction result.

S1044, carrying out weighted summation on the first prediction result and the second prediction result to obtain a final kernel value of the processor; and determines the actual number of containers based on the final core value of the processor.

In some embodiments of the present invention, the server may perform weighted summation on the first prediction result and the second prediction result to obtain a final kernel value of the processor; and the preset number of the final kernel value is used as the actual container number.

Illustratively, the final core value of the processor = a y ₁ +b*y ₂ Wherein, y ₁ Is the first prediction, y ₂ Is the second prediction, a and b are y ₁ And y ₂ The weighting coefficients, a and b, satisfy a + b =1. The predetermined number is typically one-half of the final kernel value.

It should be noted that the values of a and b are updated each time the model is trained (RMSE is used as a loss function, that is, the ab value combination that minimizes RMSE is obtained).

It can be understood that the server can respectively preprocess the first-month historical resource data and the first-week historical resource data to obtain month characteristic data and week characteristic data; inputting the monthly feature data into a preset prediction model for prediction to obtain a first prediction result; inputting the week characteristic data into a preset prediction model for prediction to obtain a second prediction result; carrying out weighted summation on the first prediction result and the second prediction result to obtain a final kernel value of the processor; and the actual number of containers is determined based on the final core number value of the processor, so that the actual number of containers can be predicted, the subsequent determination of the number of containers for capacity reduction is facilitated, and the accuracy of the container capacity reduction is improved.

And S105, carrying out capacity reduction on the second target container according to the actual container number and the acquired real-time container number.

In some embodiments of the invention, the server may obtain a real-time container number; performing difference operation according to the actual container number and the real-time container number to determine the number of the containers for capacity reduction; and based on the second target container, deleting the containers corresponding to the number of the containers to finish the capacity reduction.

It can be understood that, the server may obtain a first resource utilization rate corresponding to each of the plurality of containers, and determine a first target container according to the first resource utilization rate; a container range to be shrunk can be obtained; according to a second resource utilization rate corresponding to the target application corresponding to the first target container; screening through a second resource utilization rate and a preset container setting rule to determine a second target container; the range of the container to be shrunk can be reduced, and the accuracy and efficiency of shrinking the container are improved; according to the first historical resource data of the target application, the actual container number is predicted through a preset prediction model, and the second target container is subjected to capacity reduction according to the actual container number and the acquired real-time container number, so that the containers with low resource utilization rate can be reduced, and the utilization rate of the whole container resources is improved.

In some embodiments of the present invention, S105 may be implemented by S1051, S1052 and S1053 as follows:

s1051, acquiring the number of real-time containers.

S1052, performing difference operation according to the actual container number and the real-time container number to determine the number of the containers of the reduced volume.

In some embodiments of the present invention, the server may perform a difference operation according to the actual number of containers and the real-time number of containers to obtain an operation result; and taking the operation result as the number of the containers of the miniature.

And S1053, based on the second target container, deleting the containers corresponding to the number of the containers, and completing the capacity reduction.

In some embodiments of the present invention, the server may sum the scores applied to the containers, the scores of the containers themselves, and the difference between the actual kernel value of the container and the kernel value under an ideal condition, to obtain a container comprehensive score corresponding to the second target container; and deleting low-grade containers corresponding to the number of the containers according to the comprehensive scores of the containers to finish the capacity reduction.

It is understood that the server may obtain the real-time container number; performing difference operation according to the actual container number and the real-time container number to determine the number of the containers for capacity reduction; based on the second target container, deleting the containers corresponding to the number of the containers to finish the capacity reduction; the containers with low resource utilization rate are reduced, and the resource utilization rate of the whole container group is improved.

In some embodiments of the present invention, S106 and S107 are also performed before S104, as follows:

and S106, acquiring third history resource data and the number of the history used processor cores of the first application.

In some embodiments of the invention, the first application characterizes applications other than the target application to which the plurality of containers correspond.

S107, preprocessing the third history resource data to obtain history characteristic data; and training the initial prediction model by using the historical characteristic data to determine a preset prediction model.

In some embodiments of the invention, the historical feature data comprises a first historical feature data and a second historical feature data.

In some embodiments of the present invention, the server may determine the initial prediction model as a first initial prediction model and a second initial prediction model according to different parameter values; inputting the first historical characteristic data into an initial first prediction model, performing regression function fitting training, and determining the first prediction model; and inputting the second historical characteristic data into the initial second prediction model, performing regression function fitting training, and determining the second prediction model.

It is to be understood that the server may obtain third historical resource data and historical use processor core number of the first application; preprocessing the third history resource data to obtain history characteristic data; the initial prediction model is trained through the historical characteristic data, the preset prediction model is determined, and the prediction accuracy of the preset prediction model can be improved.

In some embodiments of the present invention, the training of the initial prediction model by using the historical feature data in S107 and the determination of the preset prediction model may be implemented through S1071, S1072, and S1073, as follows:

s1071, according to different parameter values, the initial prediction model is determined to be a first initial prediction model and a second initial prediction model.

In some embodiments of the invention, the server may determine the initial prediction model as the first initial prediction model and the second initial prediction model according to different parameter values.

S1072, inputting the first historical characteristic data into the first initial prediction model, performing regression function fitting training, and determining the first prediction model.

In some embodiments of the invention, the first historical characteristic data characterizes historical characteristic data of daily capacity; the first predictive model characterizes the prediction as used for daily capacity.

In some embodiments of the present invention, the server may input the first historical feature data into the first initial prediction model, and perform fitting training on a regression function to obtain a first fitting function; and comparing the output result of the first fitting function with a first preset result until the output result of the first fitting function reaches the first preset result, and stopping training to obtain a first prediction model.

S1073, inputting the second historical characteristic data into a second initial prediction model, performing regression function fitting training, and determining the second prediction model.

In some embodiments of the invention, the second predictive model characterizes predictions for large capacity; the second historical characteristic data characterizes historical characteristic data of the large capacity.

In some embodiments of the present invention, the server may input the second historical feature data into the second initial prediction model, and perform fitting training of the regression function to obtain a second fitting function; and comparing the output result of the second fitting function with a second preset result until the output result of the second fitting function reaches the second preset result, and stopping training to obtain a second prediction model.

It is understood that the server may determine the initial prediction model as the first initial prediction model and the second initial prediction model according to different parameter values; inputting the first historical characteristic data into a first initial prediction model, performing regression function fitting training, and determining a first prediction model; the accuracy of the first prediction model can be improved; inputting the second historical characteristic data into a second initial prediction model, performing regression function fitting training, and determining a second prediction model; the accuracy of the second prediction model may be improved.

An optional flow diagram of the container adjustment method provided in the embodiment of the present invention is shown in fig. 6, where the container adjustment method includes the following steps:

s1, obtaining the processor utilization rate and the memory utilization rate corresponding to each of the containers.

In some embodiments of the present invention, the plurality of containers each correspond to a processor usage and a memory usage (corresponding to a second historical processor usage and a second historical memory usage).

And S2, determining the first resource utilization rate through the processor utilization rate and the memory utilization rate.

And S3, screening the first resource utilization rate, and determining an initial capacity reduction container list.

In some embodiments of the present invention, the containers in the initial miniature container list (equivalent to the first target container) are.

And S4, screening the initial reduced-capacity container list through a preset machine room setting criterion, a newly online application and a periodically changed application to obtain a reduced-capacity container list.

In some embodiments of the invention, the containers in the list of containers are condensed (equivalent to the second target container).

In some embodiments of the present invention, fig. 7 is an optional flowchart illustrating a sixth method for adjusting a container, as shown in fig. 7, S4 may be implemented by S401-S404, as follows:

s401, acquiring the target application corresponding to the container in the initial capacity reduction container list to form a capacity reduction application list.

S402, calculating a second resource utilization rate of each target application in the abbreviated application list.

And S403, removing the container where the target application with the high second resource utilization rate is located from the initial reduced capacity container list to obtain an intermediate reduced capacity container list.

In some embodiments of the invention, the containers in the intermediate miniature container list (equivalent to the second initial target container) are reduced.

S404, machine room information of the containers in the intermediate capacity reduction container list is obtained, and the intermediate capacity reduction container list is screened according to preset machine room setting rules and the machine room information to obtain the capacity reduction container list.

And S5, acquiring the historical processor utilization rate and the historical memory utilization rate of the application corresponding to the container in the scaled container list.

In some embodiments of the invention, historical processor usage and historical memory usage (corresponding to the first historical resource data) are used.

And S6, preprocessing the utilization rate of the historical processor and the utilization rate of the historical memory to obtain historical characteristic data.

And S7, predicting the historical characteristic data through a preset prediction model to obtain an actual kernel number.

And S8, acquiring the number of the real-time containers.

In some embodiments of the invention, S5 and S8 are performed after S4, not in sequential order.

S9, determining the actual number of containers according to the actual number of the cores; and determining the number of the reduced containers according to the actual number of the containers and the real-time number of the containers.

And S10, carrying out capacity reduction based on the capacity reduction container list and the number of the capacity reduction containers.

It can be understood that the server may determine the initial list of miniature containers according to the screening of the first resource usage; the range of the container to be shrunk can be reduced, and the accuracy and efficiency of shrinking the container are improved; screening the initial reduced-capacity container list through a preset machine room setting criterion, a new online application and a periodically-changed application to obtain a reduced-capacity container list; further reducing the range of the containers to be shrunk, predicting historical characteristic data through a preset prediction model to obtain an actual kernel number, and determining the actual number of the containers based on the actual kernel number; and (4) according to the actual container number and the acquired real-time container number, the second target container is subjected to capacity reduction, so that the containers with low resource utilization rate can be reduced, and the resource utilization rate of the whole container is improved.

Based on the container adjusting method in the foregoing embodiment, an embodiment of the present invention further provides a container adjusting apparatus, as shown in fig. 8, fig. 8 is a schematic structural diagram of the container adjusting apparatus provided in the embodiment of the present invention, where the container adjusting apparatus 8 includes: an acquisition unit 801, a determination unit 802, a prediction unit 803, and a capacity reduction unit 804;

the obtaining unit 801 is configured to obtain first resource utilization rates corresponding to multiple containers; wherein the plurality of containers is present in at least one machine room; a container of the plurality of containers is for a virtual resource running on a server; determining a first target container with the first resource utilization rate smaller than a first preset threshold value from the plurality of containers; acquiring a target application corresponding to the first target container;

the determining unit 802 is configured to obtain a second resource usage rate corresponding to each target application; screening according to the second resource utilization rate and a preset container setting rule to determine a second target container; the preset container setting rule represents that the number of containers in which the same application is located at least reaches a preset number in a machine room;

the prediction unit 803 is configured to obtain first historical resource data of the target application; predicting the actual container number through a preset prediction model according to the first historical resource data; the preset prediction model representation is used for predicting future resource data according to historical resource data;

the capacity reduction unit 804 is configured to reduce the capacity of the second target container according to the actual number of containers and the acquired real-time number of containers.

In some embodiments of the present invention, the obtaining unit 801 is further configured to obtain second historical resource data corresponding to each of the plurality of containers;

the determining unit 802 is further configured to determine the first resource usage rates corresponding to the multiple containers by clustering the second historical resource data.

In some embodiments of the invention, the second historical resource data comprises: a second historical processor utilization and a second historical memory utilization; the determining unit 802 is further configured to cluster containers corresponding to the second historical processor utilization and the second historical memory utilization, and determine a cluster center of the container; and determining the first resource utilization rates corresponding to the multiple containers according to the cluster-like center, the second historical processor utilization rate and the distribution condition of the second historical memory utilization rate.

In some embodiments of the present invention, the obtaining unit 801 is further configured to obtain a third processor utilization rate, a third memory utilization rate, and a timing fluctuation characteristic corresponding to each target application; and calculating the second resource utilization rate corresponding to each target application according to the third processor utilization rate, the third memory utilization rate and the time sequence fluctuation characteristics.

In some embodiments of the present invention, the determining unit 802 is further configured to determine, from a plurality of target applications, a first application of which the second resource usage rate is greater than a second preset threshold; wherein the first application characterizes a normally operating application; based on the first target container, removing containers corresponding to the first application, the newly online application and the periodically changed application, and determining a second initial target container;

the obtaining unit 801 is further configured to obtain machine room information corresponding to the second initial target container;

the determining unit 802 is further configured to filter the second initial target container according to the preset container setting rule and the machine room information, and determine the second target container.

In some embodiments of the present invention, the obtaining unit 801 is further configured to divide the second initial target container according to the machine room information to obtain a plurality of container groups; wherein the container group represents a container in a machine room;

the determining unit 802 is further configured to determine, for the multiple container groups, the fixed containers corresponding to the multiple container groups respectively according to the preset container setting rule; and based on the second initial target container, rejecting the fixed container and determining the second target container.

In some embodiments of the present invention, before predicting the actual number of containers through a preset prediction model according to the first historical resource data, the obtaining unit 801 is further configured to obtain third historical resource data and a historical used processor core number of the first application; wherein the first application characterizes applications other than the target application corresponding to the plurality of containers;

the determining unit 802 is further configured to preprocess the third history resource data to obtain history feature data; and training an initial prediction model by using the historical characteristic data to determine the preset prediction model.

In some embodiments of the invention, the historical feature data comprises: first and second historical feature data; the preset prediction model comprises: a first predictive model and a second predictive model; the determining unit 802 is further configured to determine the initial prediction model as a first initial prediction model and a second initial prediction model according to different parameter values; inputting the first historical characteristic data into a first initial prediction model, performing regression function fitting training, and determining the first prediction model; wherein the first historical characteristic data characterizes historical characteristic data of daily capacity; the first prediction model represents the prediction used in daily capacity, the second historical characteristic data is input into a second initial prediction model, regression function fitting training is carried out, and the second prediction model is determined; wherein the second historical characteristic data characterizes historical characteristic data of the high lift capacity and the second predictive model characterizes predictions for the high lift capacity.

In some embodiments of the invention, the first historical resource data comprises: first month historical resource data and first week historical resource data; the obtaining unit 801 is further configured to perform preprocessing on the first month historical resource data and the first week historical resource data respectively to obtain month feature data and week feature data; inputting the month characteristic data into the preset prediction model for prediction to obtain a first prediction result; the first prediction result represents a core value of a processor corresponding to the lunar data; inputting the week characteristic data into the preset prediction model for prediction to obtain a second prediction result; the second prediction result represents a kernel value of a processor corresponding to the week data; carrying out weighted summation on the first prediction result and the second prediction result to obtain a final kernel value of the processor; and determining the actual number of containers based on the final core value of the processor.

In some embodiments of the present invention, the obtaining unit 801 is further configured to obtain the real-time container number;

the determining unit 802 is further configured to perform a difference operation according to the actual number of containers and the real-time number of containers, and determine the number of containers for capacity reduction;

the capacity reduction unit 804 is further configured to delete the containers corresponding to the number of containers based on the second target container, so as to complete capacity reduction.

In addition, when the container adjustment is performed, only the division of each program module is illustrated, and in practical applications, the processing allocation may be completed by different program modules as needed, that is, the internal structure of the apparatus may be divided into different program modules to complete all or part of the processing described above. In addition, the embodiments of the container adjusting device and the container adjusting method provided in the above embodiments belong to the same concept, and specific implementation processes and beneficial effects thereof are described in detail in the embodiments of the method, and are not described again here. For technical details not disclosed in the embodiments of the apparatus, reference is made to the description of the embodiments of the method of the invention for understanding.

Based on the container adjusting method in the foregoing embodiment, an embodiment of the present invention further provides a container adjusting apparatus, as shown in fig. 9, where fig. 9 is a schematic structural diagram of a container adjusting apparatus provided in an embodiment of the present invention, and the apparatus 9 includes: a processor 901 and a memory 902; the memory 902 stores one or more programs executable by the processor, and when the one or more programs are executed, the processor 901 performs any one of the container adjustment methods according to the foregoing embodiments.

Embodiments of the present invention also provide a computer-readable storage medium storing executable instructions, which when executed, are configured to cause a processor to execute the container adjustment method according to an embodiment of the present invention.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.

Claims

1. A method of container adjustment, comprising:

acquiring a first resource utilization rate corresponding to each of a plurality of containers; wherein the plurality of containers is present in at least one machine room; a container of the plurality of containers is for a virtual resource running on a server;

determining a first target container with the first resource utilization rate smaller than a first preset threshold value from the plurality of containers; acquiring a target application corresponding to the first target container;

acquiring a second resource utilization rate corresponding to each target application; screening according to the second resource utilization rate and a preset container setting rule to determine a second target container; the preset container setting rule represents that the number of containers in which the same application is located at least reaches a preset number in a machine room;

acquiring first historical resource data of the target application; predicting the actual container number through a preset prediction model according to the first historical resource data; the preset prediction model representation is used for predicting future resource data according to historical resource data;

and carrying out capacity reduction on the second target container according to the actual container number and the acquired real-time container number.

2. The method according to claim 1, wherein the obtaining the first resource usage rate corresponding to each of the plurality of containers comprises:

acquiring second historical resource data corresponding to the containers respectively;

and determining the first resource utilization rate corresponding to each of the plurality of containers by clustering the second historical resource data.

3. The method of claim 2, wherein the second historical resource data comprises: a second history processor utilization and a second history memory utilization;

the determining the first resource usage rates corresponding to the plurality of containers by clustering the second historical resource data includes:

clustering containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate, and determining a cluster center of the containers;

and determining the first resource utilization rates corresponding to the containers according to the distribution conditions of the cluster center, the second historical processor utilization rate and the second historical memory utilization rate.

4. The method according to claim 1, wherein the obtaining the second resource usage rate corresponding to each target application comprises:

acquiring the utilization rate of a third processor, the utilization rate of a third memory and time sequence fluctuation characteristics corresponding to each target application;

and calculating the second resource utilization rate corresponding to each target application according to the third processor utilization rate, the third memory utilization rate and the time sequence fluctuation characteristics.

5. The method according to claim 1, wherein the screening to determine the second target container according to the second resource usage rate and a preset container setting rule comprises:

determining a first application with the second resource utilization rate larger than a second preset threshold value from a plurality of target applications; wherein the first application characterizes a normally functioning application;

based on the first target container, removing containers corresponding to the first application, the newly online application and the periodically changed application, and determining a second initial target container;

acquiring machine room information corresponding to the second initial target container;

and screening the second initial target container according to the preset container setting rule and the machine room information to determine the second target container.

6. The method according to claim 5, wherein the screening the second initial target container according to the preset container setting rule and the machine room information to determine the second target container comprises:

dividing the second initial target container according to the machine room information to obtain a plurality of container groups; wherein the container group represents a container in a machine room;

for the plurality of container groups, respectively determining the fixed containers corresponding to the plurality of container groups through the preset container setting rules;

and based on the second initial target container, rejecting the fixed container and determining the second target container.

7. The method according to any one of claims 1-6, wherein before predicting the actual number of containers from the first historical resource data by a predetermined prediction model, the method further comprises:

acquiring third history resource data and history use processor core number of the first application; wherein the first application characterizes applications other than the target application corresponding to the plurality of containers;

preprocessing the third history resource data to obtain history characteristic data; and training an initial prediction model by using the historical characteristic data to determine the preset prediction model.

8. The method of claim 7, wherein the historical characterization data comprises: first and second historical feature data; the preset prediction model comprises: a first predictive model and a second predictive model;

the training of the initial prediction model by using the historical characteristic data to determine the preset prediction model comprises the following steps:

determining the initial prediction model as a first initial prediction model and a second initial prediction model according to different parameter values;

inputting the first historical characteristic data into a first initial prediction model, performing regression function fitting training, and determining the first prediction model; wherein the first historical characteristic data characterizes historical characteristic data of daily capacity; the first predictive model characterizes predictions for daily capacity

Inputting the second historical characteristic data into a second initial prediction model, performing regression function fitting training, and determining the second prediction model; wherein the second historical characteristic data characterizes historical characteristic data of the high lift capacity and the second predictive model characterizes predictions for the high lift capacity.

9. The method of claim 1, wherein the first historical resource data comprises: first month historical resource data and first week historical resource data;

the predicting the actual number of containers according to the first historical resource data through a preset prediction model comprises the following steps:

respectively preprocessing the first month historical resource data and the first week historical resource data to obtain month characteristic data and week characteristic data;

inputting the month characteristic data into the preset prediction model for prediction to obtain a first prediction result; the first prediction result represents a core value of a processor corresponding to the lunar data;

inputting the week characteristic data into the preset prediction model for prediction to obtain a second prediction result; the second prediction result represents a core value of a processor corresponding to the week data;

carrying out weighted summation on the first prediction result and the second prediction result to obtain a final kernel value of the processor; and determining the actual number of containers based on the final core value of the processor.

10. The method according to any one of claims 1 to 6, wherein the contracting the second target container according to the actual number of containers and the obtained real-time number of containers comprises:

acquiring the real-time container number;

performing difference operation according to the actual container number and the real-time container number to determine the number of the reduced containers;

and deleting the containers corresponding to the number of the containers based on the second target container to finish the capacity reduction.

11. A container adjusting device is characterized by comprising an obtaining unit, a determining unit, a predicting unit and a capacity reducing unit; wherein the content of the first and second substances,

the determining unit is configured to obtain a second resource usage rate corresponding to each target application; screening according to the second resource utilization rate and a preset container setting rule to determine a second target container; the preset container setting rule represents that the number of containers in which the same application is located at least reaches a preset number in a machine room;

12. A container conditioning apparatus, comprising:

a memory for storing executable instructions;

a processor for implementing the container adaptation method of any of claims 1-10 when executing executable instructions stored in the memory.

13. A computer-readable storage medium having stored thereon executable instructions for causing a processor to perform the container adjustment method of any one of claims 1-10 when the executable instructions are executed.