CN115390995A - Method, device, equipment and medium for adjusting number of containers - Google Patents
Method, device, equipment and medium for adjusting number of containers Download PDFInfo
- Publication number
- CN115390995A CN115390995A CN202211165647.0A CN202211165647A CN115390995A CN 115390995 A CN115390995 A CN 115390995A CN 202211165647 A CN202211165647 A CN 202211165647A CN 115390995 A CN115390995 A CN 115390995A
- Authority
- CN
- China
- Prior art keywords
- parameters
- containers
- monitoring data
- link monitoring
- container
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
Abstract
The embodiment of the application discloses a method, a device, equipment and a medium for adjusting the number of containers, which are used for acquiring full-link monitoring data; the full link monitoring data includes container operating parameters and service parameters, which may be used to characterize service invocation. And constructing a correlation parameter for representing the correlation between the number of the containers and the service calling condition according to the container operation parameter and the service parameter. And predicting the container quantity based on the full link monitoring data and the associated parameters to obtain a container quantity estimated value. The number of containers in the system is adjusted based on the estimated number of containers. The acquired parameters are relatively comprehensive by acquiring different types of parameters and mining the relevance among the parameters to obtain the relevant parameters, so that the estimated value of the number of the containers meeting the service requirement is relatively accurately obtained.
Description
Technical Field
The present application relates to the field of container service technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for adjusting a number of containers.
Background
The container technology does not need to virtualize the whole operating system, only needs to virtualize a small-scale environment, has high starting speed, and basically does not consume additional system resources except for running the application in the container. Docker is the most widely used container technology, and creates a service by packaging images and starting containers. However, as applications become more complex, the number of containers also increases, thereby deriving a significant problem in managing the operation and maintenance of the containers, and k8s (kubernets) has come into play. k8s is an orchestration management tool for a portable container generated for container services.
At present, a k8s built-in HPA (Horizontal Pod Autoscaler) usually depends on a single type of parameter to adjust the number of containers when controlling the elastic expansion and contraction of the containers. For example, when the usage amount of a CPU (Central Processing Unit) or the usage amount of a memory is high, in order to reduce the operating pressure of the background service, the capacity of the container may be expanded, that is, the number of containers may be increased. The single type of index cannot comprehensively reflect the operating pressure of the background service, so that the number of the adjusted containers does not meet the actual service requirement. When the number of the containers is adjusted to be higher, the utilization rate of the containers is lower, and resource waste is caused. When the number of containers is adjusted to be low, the operation pressure of the background cannot be relieved.
Therefore, how to timely and accurately realize the elastic expansion and contraction of the number of the containers is a problem to be solved by the technical personnel in the field.
Disclosure of Invention
The embodiment of the application aims to provide a method, a device, equipment and a computer-readable storage medium for adjusting the number of containers, which can timely and accurately realize the elastic expansion and contraction of the number of containers.
In order to solve the above technical problem, an embodiment of the present application provides a method for adjusting the number of containers, including:
acquiring full link monitoring data; wherein the full link monitoring data comprises container operation parameters and service parameters; the service parameters are used for representing service calling conditions;
constructing a correlation parameter according to the container operation parameter and the service parameter; the correlation parameters are used for representing the correlation between the number of containers and the service calling condition;
predicting the container quantity based on the full link monitoring data and the correlation parameters to obtain a container quantity predicted value;
and adjusting the container quantity of the system according to the estimated value of the container quantity.
Optionally, the predicting the number of containers based on the full link monitoring data and the associated parameters to obtain a container number prediction value includes:
and inputting the full link monitoring data and the associated parameters into a prediction model to obtain a container quantity predicted value.
Optionally, the predictive model is a time series model;
the training process of the time series model comprises the following steps:
acquiring historical full-link monitoring data meeting periodic requirements; wherein the historical full link monitoring data comprises historical container operating parameters and historical service parameters;
according to the historical container operation parameters and the historical service parameters, historical association parameters are constructed;
and training an initial time sequence model by using the historical full link monitoring data and the historical associated parameters to obtain a time sequence model for predicting the number of containers.
Optionally, the predicting the number of containers based on the full link monitoring data and the associated parameters to obtain a container number prediction value includes:
inputting the full link monitoring data into a prediction model to obtain a container data initial prediction value;
determining a target container quantity adjusting proportion corresponding to the association parameters based on a corresponding mode of the set association parameters and the container quantity adjusting proportion;
and adjusting the initial estimated value of the container quantity according to the target container quantity adjusting proportion to obtain the estimated value of the container quantity.
Optionally, the predictive model is a time series model;
the training process of the time series model comprises the following steps:
acquiring historical full-link monitoring data meeting periodic requirements; the historical full-link monitoring data comprises historical container operation parameters and historical service parameters;
and training an initial time sequence model by using the historical full link monitoring data to obtain a time sequence model for predicting the number of containers.
Optionally, the acquiring full-link monitoring data includes:
collecting initial full link monitoring data;
cleaning the initial full-link monitoring data according to a set data cleaning mode to obtain full-link monitoring data; the data cleaning mode comprises abnormal data elimination, missing data completion and parameter normalization.
Optionally, the estimated value of the number of containers includes estimated values of the number of containers corresponding to a plurality of time periods respectively; the adjusting the number of containers of the system according to the estimated value of the number of containers comprises:
based on a target time period to which the current time of the system belongs, adjusting the container quantity of the system to a container quantity pre-estimated value corresponding to the target time period; wherein the target time period is any one of the plurality of time periods.
The embodiment of the application also provides a device for adjusting the number of containers, which comprises an acquisition unit, a construction unit, a prediction unit and an adjustment unit;
the acquisition unit is used for acquiring full link monitoring data; wherein the full link monitoring data comprises container operation parameters and service parameters; the service parameters are used for representing service calling conditions;
the construction unit is used for constructing correlation parameters according to the container operation parameters and the service parameters; the correlation parameters are used for representing the correlation between the container quantity and the service calling condition;
the prediction unit is used for predicting the container quantity based on the full link monitoring data and the associated parameters to obtain a container quantity predicted value;
and the adjusting unit is used for adjusting the container quantity of the system according to the container quantity estimated value.
Optionally, the prediction unit is configured to input the full-link monitoring data and the associated parameter into a prediction model to obtain a container quantity prediction value.
Optionally, the predictive model is a time series model; for a training process of the time series model, the apparatus comprises a training unit;
the acquisition unit is used for acquiring historical full-link monitoring data meeting the periodic requirement; wherein the historical full link monitoring data comprises historical container operating parameters and historical service parameters;
the construction unit is used for constructing historical association parameters according to the historical container operation parameters and the historical service parameters;
and the training unit is used for training an initial time series model by using the historical full link monitoring data and the historical association parameters so as to obtain a time series model for predicting the number of containers.
Optionally, the prediction unit includes a prediction subunit, a determination subunit, and an obtaining subunit;
the pre-estimation subunit is used for inputting the full link monitoring data into a prediction model to obtain an initial pre-estimation value of container data;
the determining subunit is configured to determine, based on a correspondence manner between the set association parameter and the container quantity adjustment ratio, a target container quantity adjustment ratio corresponding to the association parameter;
and the obtaining subunit is configured to adjust the initial estimated value of the container quantity according to the target container quantity adjustment ratio, so as to obtain a container quantity estimated value.
Optionally, the predictive model is a time series model; for a training process of the time series model, the apparatus comprises a training unit;
the acquisition unit is used for acquiring historical full-link monitoring data meeting the periodic requirement; the historical full-link monitoring data comprises historical container operation parameters and historical service parameters;
and the training unit is used for training an initial time sequence model by using the historical full link monitoring data to obtain a time sequence model for predicting the number of containers.
Optionally, the acquiring unit includes a collecting subunit and a cleaning subunit;
the acquisition subunit is used for acquiring initial full-link monitoring data;
the cleaning subunit is configured to clean the initial full-link monitoring data according to a set data cleaning manner to obtain full-link monitoring data; the data cleaning mode comprises abnormal data elimination, missing data completion and parameter normalization.
Optionally, the estimated value of the number of containers includes estimated values of the number of containers corresponding to a plurality of time periods respectively; the adjusting unit is used for adjusting the container quantity of the system to a container quantity estimated value corresponding to a target time period based on the target time period to which the current time of the system belongs; wherein the target time period is any one of the plurality of time periods.
The embodiment of the present application further provides an adjusting apparatus for a number of containers, including:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the method for adjusting the number of containers as described above.
The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for adjusting the number of containers are implemented as described above.
According to the technical scheme, the full link monitoring data is obtained; the full link monitoring data comprises container operation parameters and service parameters, and the service parameters can be used for representing service calling conditions. Compared with a single type of parameter, the full link monitoring data contains more comprehensive parameter types. The more comprehensive the parameters associated with the containers, the more accurately the number of containers needed for service can be predicted. In order to fully mine the relevance among the parameters, the relevance parameters can be constructed according to the container operation parameters and the service parameters; the correlation parameter is used for characterizing the correlation between the number of the containers and the service calling condition. A container number estimate may be obtained by predicting the container number based on the full link monitoring data and the associated parameters. Based on the estimated number of containers, the number of containers in the system can be adjusted. In the technical scheme, the parameters of different types are obtained, and the relevance among the parameters is mined to obtain the relevant parameters, so that the obtained parameters are relatively comprehensive, and the estimated value of the number of containers meeting the service requirement can be relatively accurately obtained based on the parameters.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a schematic view of a scenario for adjusting the number of containers according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a method for adjusting the number of containers according to an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an apparatus for adjusting the number of containers according to an embodiment of the present disclosure;
fig. 4 is a structural diagram of an apparatus for adjusting the number of containers according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort belong to the protection scope of the present application.
The terms "including" and "having," and any variations thereof in the description and claims of this application and the above-described drawings, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements but may include other steps or elements not expressly listed.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings.
In a traditional mode, capacity expansion of a container is usually performed by increasing the number of containers to reduce the pressure of a background server under the condition that the operating pressure of the background server is high. However, this method belongs to post-expansion, and the expansion is performed only when the pressure of the background server is high, so that the server has the problem of slow response in the period before the expansion. There are many factors that cause the background server to be stressed, but the traditional capacity expansion mode often depends on the usage of a CPU or a memory to adjust the number of containers, which may cause that the adjusted number of containers may not meet the actual service requirement.
Therefore, the embodiment of the application provides a method, a device, equipment and a computer readable storage medium for adjusting the number of containers, and full link monitoring data is obtained; the full link monitoring data comprises container operation parameters and service parameters, and the service parameters can be used for representing service calling conditions. Constructing a correlation parameter according to the container operation parameter and the service parameter; the association parameter is used for characterizing the relevance of the container number and the service calling condition. The container quantity is predicted based on the full link monitoring data and the associated parameters, and a container quantity predicted value can be obtained, so that the container quantity of the system is adjusted according to the container quantity predicted value, and the container quantity is predicted in advance.
The adjusting scheme of the container number provided by the embodiment of the application can be suitable for a pass platform, wherein the pass platform is a platform which is formed by taking k8s + docker as a core and has organization capability. Wherein, the container (pod) is the smallest scheduling unit in k8s, and is also the smallest unit for management and creation, and the pod in k8s provides service for the outside.
Fig. 1 is a schematic view of a scenario for adjusting the number of containers according to an embodiment of the present disclosure, where the work that a server needs to execute may be divided into three modules, which are a full-link monitoring module, a prediction module (adapter), and a container number adjusting module (HPA module). The full link monitoring module can acquire full link monitoring data; the full link monitoring data comprises container operation parameters and service parameters, and the service parameters can be used for representing service calling conditions. The full link monitoring module may be implemented using a Prometheus component. And constructing the correlation parameters according to the container operation parameters and the service parameters. Compared with a single type of parameter, the full link monitoring data contains more comprehensive parameter types. The more comprehensive the parameters associated with the containers, the more accurately the number of containers needed for service can be predicted. In order to fully mine the relevance among the parameters, the relevance parameters are added on the basis of full-link monitoring data.
Considering that the variation of the service demand on the server is always periodic, the prediction module can analyze the full-link monitoring data and the associated parameters by combining the correlation between each parameter and the container number in different time ranges, so as to obtain the container number prediction value. The container quantity adjusting module can adjust the container quantity of the system according to the container quantity estimated value. The parameters of different types are obtained, and the relevance among the parameters is mined to obtain the relevant parameters, so that the obtained parameters are relatively comprehensive, and the estimated value of the number of containers meeting the service requirement can be relatively accurately obtained based on the parameters.
Next, a method for adjusting the number of containers provided in the embodiments of the present application will be described in detail. Fig. 2 is a flowchart of a method for adjusting the number of containers according to an embodiment of the present disclosure, where the method includes:
s201: and acquiring full link monitoring data.
The full link monitoring data may include container operation parameters and service parameters, among others.
In order to more fully understand the operating pressure of the server, the container operating parameters and the service parameters may each include various types of data.
In a specific implementation, the container operation parameters may include a current cpu usage c (t), a memory usage m (t), a disk block IO usage b (t), a network card data amount n (t), and the like. The service parameters may be used to characterize service invocation conditions, and may include the number of times each service is invoked in each period of time, time consumed for each invocation, invocation time point, and the like.
For the acquisition of the full-link monitoring data, the full-link monitoring data can be acquired every 10 minutes for all containers in the cluster through the monitoring platform. In the subsequent determination of the estimated number of containers, this may be performed every 1 hour.
In consideration of practical applications, the collected data may have irregular situations, for example, invalid data, abnormal data, or data incompletion may occur. Therefore, in the embodiment of the application, initial full-link monitoring data can be collected; cleaning the initial full-link monitoring data according to a set data cleaning mode to obtain full-link monitoring data; the data cleaning mode comprises abnormal data elimination, missing data completion and parameter normalization.
In specific implementation, for the case that part of data occurring in the acquisition process is abnormal, abnormal data can be removed through a 3sigma principle. Meanwhile, for the condition that partial data is missing, a K neighbor interpolation method can be adopted to complement the missing data. As the acquired data are data with different multidimensional characteristics, in order to realize the subsequent analysis of the data, the data need to be changed into dimensionless normalized data, and the full-link monitoring data is mapped between (0, 1) by transforming the full-link monitoring data.
The data normalization processing mode is conventional, and reference may be made to the existing technology specifically, and details are not described here again.
S202: and constructing the correlation parameters according to the container operation parameters and the service parameters.
The more comprehensive the parameters related to the containers, the more accurate the prediction of the number of containers required for the service. In the embodiment of the application, in order to fully mine the relevance between the parameters, the relevance parameters can be constructed according to the container operation parameters and the service parameters.
The correlation parameter is used for characterizing the correlation between the number of the containers and the service calling condition.
Taking the above-described 7 types of data, i.e., the current cpu usage c (t), the memory usage m (t), the disk block IO usage b (t), the network card data amount n (t), the number of times of calling each service in each period of time, and the time consumed by calling each time, as an example, can be represented by x1 (t) to x7 (t) in sequence.
Considering that in practical application, there is a certain correlation between the current cpu usage c (t) and the number of calls in each period of time corresponding to each service, so that the cross-correlation between the two indexes can be calculated to obtain a correlation parameter denoted as x8 (t), and the specific calculation method may be to calculate the covariance of the two indexes, and the specific formula is:
wherein x1 (t) represents the cpu usage amount at t time, x5 (t) represents the number of calls corresponding to each service at t time,
represents the average of the usage amounts corresponding to all cpus,
represents the average number of calls corresponding to all services, and n represents the number of cpus.
From this covariance it can be estimated whether the service is CPU intensive. When the covariance value is positive, the service is CPU intensive, the response time of the service can be obviously reduced by increasing the number of the pod, and the problem of stuck is solved. And when the value is larger, it is said that increasing the pod number decreases the influence time of the service.
When the covariance value is negative, the service is not CPU-intensive, the response time can be improved to a certain extent by increasing the pod number, and the deadlock problem of the service can be alleviated, but the effect is not obvious because the I/O processing of the bottleneck service of the service is not on the CPU.
S203: and predicting the container quantity based on the full link monitoring data and the associated parameters to obtain a container quantity estimated value.
Considering that the variation of the service demand on the server often has periodicity, in the embodiment of the present application, when predicting the number of containers based on the full link monitoring data and the associated parameters, the full link monitoring data and the associated parameters may be analyzed in combination with the correlations between the parameters and the number of containers in different time ranges, so as to estimate the number of containers.
In specific implementation, a prediction model capable of reflecting the correlation between each parameter and the number of containers in different time ranges can be obtained through a model training mode, so that full-link monitoring data and associated parameters can be input into the prediction model to obtain a container number estimated value.
The prediction model may employ a time series model. For the training process of the time series model, the training process can be obtained by depending on historical full link monitoring data of a server.
In specific implementation, historical full-link monitoring data meeting periodic requirements can be obtained; the historical full link monitoring data may include, among other things, historical container operating parameters and historical service parameters. And constructing historical associated parameters according to the historical container operation parameters and the historical service parameters.
The periodic requirement can be the limitation on the collection time span of the historical full-link monitoring data, and the longer the time span is, the more the data quantity of the collected historical full-link monitoring data is, and the more the training of the model is facilitated. And the time span of the historical full-link monitoring data is limited, and the historical full-link monitoring data in the latest period of time can be collected, so that the variation trend of the operating pressure of the server predicted by the trained time series model is more suitable for the actual operating condition of the current server.
The implementation manner of constructing the history association parameter may refer to the step of S202, which is not described herein again.
After the historical container operation parameters and the historical service parameters are obtained and the historical association parameters are constructed, the initial time series model can be trained by using the historical full link monitoring data and the historical association parameters to obtain a time series model for predicting the number of containers.
After the training of the time series model is completed, the trained time series model is directly called subsequently, and the time series model is obtained without training each time the adjustment of the number of containers is executed.
S204: the number of containers in the system is adjusted based on the estimated number of containers.
Taking the time series model as an example, when the time series model is obtained through training, the output form of the time series model may be defined, for example, full link monitoring data and associated parameters are input to the time series model, and a container quantity estimated value may be output. Full link monitoring data and associated parameters may also be input to the time series model, and a plurality of bin quantity estimates may be output, each having its applicable time range.
Taking the example of outputting a container quantity estimated value, in practical application, the container quantity estimated value is directly used as the container quantity of the server system in the next period of time.
Taking the container quantity estimated value including the container quantity estimated values corresponding to the multiple time periods as an example, in practical application, the container quantity of the system may be adjusted to the container quantity estimated value corresponding to the target time period based on the target time period to which the current time of the system belongs. The target time period may be any one of a plurality of time periods.
For example, assuming that the current time is 8 am, the predicted values of the number of containers obtained by the above operations are 100 and 130, wherein 100 corresponds to a time period of 8 am to 8 am for 30 minutes, and 130 corresponds to a time period of 8 am to 9 am for 30 minutes. In a particular implementation, the number of containers for the system may be adjusted to 100 at 8 am and 130 at 30 am. At the time 9 am, the predicted value of the number of containers corresponding to the next time may be predicted again according to the operations of S201 to S204 described above.
For convenience of description, the container operation parameters, the service parameters and the associated parameters may be collectively referred to as parameters, in the embodiment of the present application, the currently obtained parameters are analyzed depending on the correlation between each parameter and the number of containers in different time ranges, the number of containers required for service in the next period of time may be predicted based on the current parameters, advance prediction of the number of containers is realized, and thus elastic expansion and contraction of the number of containers is realized in time and accurately depending on the container number predicted value.
According to the technical scheme, the full link monitoring data is obtained; the full link monitoring data comprises container operation parameters and service parameters, and the service parameters can be used for representing service calling conditions. Compared with a single type of parameter, the full link monitoring data comprises more comprehensive parameter types. The more comprehensive the parameters related to the containers, the more accurate the prediction of the number of containers required for the service. In order to fully mine the relevance among the parameters, the relevance parameters can be constructed according to the container operation parameters and the service parameters; the correlation parameter is used for characterizing the correlation between the number of the containers and the service calling condition. A container number estimate may be obtained by predicting the container number based on the full link monitoring data and the associated parameters. Based on the estimated number of containers, the number of containers in the system can be adjusted. In the technical scheme, the association parameters are obtained by acquiring the parameters of different types and mining the association among the parameters, so that the acquired parameters are relatively comprehensive, and the estimated value of the number of containers meeting the service requirement can be relatively accurately obtained based on the parameters.
In the above description, the time series model is used to analyze the full link monitoring data and the associated parameters at the same time to obtain the estimated value of the number of containers. In addition, in the embodiment of the application, the full-link monitoring data can also be input into a prediction model to obtain an initial estimation value of the container data; determining a target container quantity adjusting proportion corresponding to the association parameters based on a corresponding mode of the set association parameters and the container quantity adjusting proportion; and adjusting the initial estimated value of the container quantity according to the target container quantity adjusting proportion to obtain the estimated value of the container quantity.
By combining the above description, if the value of the associated parameter is positive, the response time of the service can be significantly reduced by increasing the pod number; when the value of the correlation parameter is negative, adjusting the pod number has no obvious effect on reducing the service response time.
The set correspondence between the correlation parameter and the adjustment ratio of the number of containers may include respective ratios corresponding to different value ranges, where the value range may include only a range in which the value of the correlation parameter is positive. After the value of the association parameter is determined, the container quantity adjustment proportion matched with the association parameter can be obtained by inquiring the corresponding mode of the association parameter and the container quantity adjustment proportion, and the container quantity adjustment proportion can be called as a target container quantity adjustment proportion for distinguishing.
Correspondingly, in order to enable the time series model to more accurately analyze the full link monitoring data, historical full link monitoring data meeting the periodicity requirement can be obtained when the training of the time series model is performed; wherein the historical full link monitoring data comprises historical container operating parameters and historical service parameters. And training the initial time series model by using historical full-link monitoring data to obtain a time series model for predicting the number of containers.
In the embodiment of the present application, a specific implementation manner for obtaining the estimated value of the number of containers is not limited, and the estimated value of the number of containers may be obtained by analyzing the full-link monitoring data and the associated parameters simultaneously according to the time sequence model, where the time sequence model is obtained by training according to the historical full-link monitoring data and the historical associated parameters. The method also can analyze the full-link monitoring data according to a time sequence model to obtain an initial estimated value of the container quantity, and then adjust the initial estimated value of the container quantity according to the associated parameters to obtain a final estimated value of the container quantity, wherein the time sequence model is obtained by training according to historical full-link monitoring data.
Fig. 3 is a schematic structural diagram of an adjusting apparatus for the number of containers according to an embodiment of the present application, including an obtaining unit 31, a constructing unit 32, a predicting unit 33, and an adjusting unit 34;
an obtaining unit 31, configured to obtain full link monitoring data; the full link monitoring data comprises container operation parameters and service parameters; the service parameters are used for representing service calling conditions;
a construction unit 32, configured to construct a correlation parameter according to the container operation parameter and the service parameter; the correlation parameters are used for representing the correlation between the container quantity and the service calling condition;
a prediction unit 33 for predicting the container number based on the full link monitoring data and the associated parameters to obtain a container number estimated value
An adjusting unit 34 is used for adjusting the number of containers of the system according to the estimated value of the number of containers.
Optionally, the prediction unit is configured to input the full-link monitoring data and the associated parameters into a prediction model to obtain a predicted value of the number of containers.
Optionally, the predictive model is a time series model; aiming at the training process of the time series model, the device comprises a training unit;
the acquisition unit is used for acquiring historical full-link monitoring data meeting the periodic requirement; the historical full-link monitoring data comprises historical container operation parameters and historical service parameters;
the construction unit is used for constructing historical associated parameters according to the historical container operation parameters and the historical service parameters;
and the training unit is used for training the initial time sequence model by utilizing the historical full link monitoring data and the historical associated parameters so as to obtain the time sequence model for predicting the number of the containers.
Optionally, the prediction unit includes a prediction subunit, a determination subunit, and an obtaining subunit;
the prediction subunit is used for inputting the full-link monitoring data into the prediction model to obtain an initial prediction value of the container data;
the determining subunit is used for determining a target container quantity adjustment ratio corresponding to the association parameter based on a corresponding mode of the set association parameter and the container quantity adjustment ratio;
and the obtaining subunit is used for adjusting the initial estimated value of the container quantity according to the target container quantity adjustment proportion to obtain the estimated value of the container quantity.
Optionally, the predictive model is a time series model;
for time series models
The apparatus comprises a training unit;
the acquisition unit is used for acquiring historical full-link monitoring data meeting the periodic requirement; the historical full-link monitoring data comprises historical container operation parameters and historical service parameters;
and the training unit is used for training the initial time sequence model by using the historical full link monitoring data to obtain a time sequence model for predicting the number of containers.
Optionally, the acquiring unit comprises a collecting subunit and a cleaning subunit;
the acquisition subunit is used for acquiring initial full-link monitoring data;
the cleaning subunit is used for cleaning the initial full-link monitoring data according to a set data cleaning mode to obtain full-link monitoring data; the data cleaning mode comprises abnormal data elimination, missing data completion and parameter normalization.
Optionally, the estimated value of the number of containers includes estimated values of the number of containers corresponding to a plurality of time periods respectively; the adjusting unit is used for adjusting the container quantity of the system to a container quantity estimated value corresponding to the target time period based on the target time period to which the current time of the system belongs; wherein the target time period is any one of a plurality of time periods.
The description of the features in the embodiment corresponding to fig. 3 may refer to the related description of the embodiment corresponding to fig. 2, and is not repeated here.
According to the technical scheme, the full link monitoring data is obtained; the full link monitoring data comprises container operation parameters and service parameters, and the service parameters can be used for representing service calling conditions. Compared with a single type of parameter, the full link monitoring data comprises more comprehensive parameter types. The more comprehensive the parameters related to the containers, the more accurate the prediction of the number of containers required for the service. In order to fully mine the relevance among the parameters, the relevance parameters can be constructed according to the container operation parameters and the service parameters; the correlation parameter is used for representing the correlation between the container number and the service calling condition. A container number estimate may be obtained by predicting the container number based on the full link monitoring data and the associated parameters. Based on the estimated number of containers, the number of containers in the system can be adjusted. In the technical scheme, the parameters of different types are obtained, and the relevance among the parameters is mined to obtain the relevant parameters, so that the obtained parameters are relatively comprehensive, and the estimated value of the number of containers meeting the service requirement can be relatively accurately obtained based on the parameters.
Fig. 4 is a structural diagram of an apparatus for adjusting the number of containers according to an embodiment of the present application, and as shown in fig. 4, the apparatus for adjusting the number of containers includes: a memory 20 for storing a computer program;
the processor 21 is configured to implement the steps of the method for adjusting the number of containers according to the above embodiment when executing the computer program.
The adjusting device for the number of containers provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, or a desktop computer.
The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may further include an AI (Artificial Intelligence) processor for processing a calculation operation related to machine learning.
The memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing a computer program 201, wherein after being loaded and executed by the processor 21, the computer program can implement the relevant steps of the method for adjusting the number of containers disclosed in any one of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202, data 203, and the like, and the storage manner may be a transient storage manner or a permanent storage manner. Operating system 202 may include, among others, windows, unix, linux, and the like. Data 203 may include, but is not limited to, correlations of various parameters with container numbers over different time ranges, full link monitoring data, correlation parameters characterizing correlations of container numbers with service invocation instances, and the like.
In some embodiments, the device for adjusting the number of containers may further include a display 22, an input/output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
It will be appreciated by those skilled in the art that the configuration shown in figure 4 does not constitute a limitation on the number of containers and may include more or fewer components than those shown.
It is to be understood that, if the method for adjusting the number of containers in the above embodiments is implemented in the form of a software functional unit and sold or used as a stand-alone product, it may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, which are essential or part of the prior art, or all or part of the technical solutions may be embodied in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic or optical disk, and other various media capable of storing program codes.
Based on this, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for adjusting the number of containers are implemented.
The functions of the functional modules of the computer-readable storage medium according to the embodiment of the present invention may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description of the foregoing method embodiment, which is not described herein again.
A method, an apparatus, a device, and a computer-readable storage medium for adjusting the number of containers provided in the embodiments of the present application are described above in detail. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
A method, an apparatus, a device and a computer-readable storage medium for adjusting the number of containers provided in the present application are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present application.
Claims (10)
1. A method for adjusting the number of containers, comprising:
acquiring full link monitoring data; wherein the full link monitoring data comprises container operation parameters and service parameters; the service parameters are used for representing service calling conditions;
constructing a correlation parameter according to the container operation parameter and the service parameter; the correlation parameters are used for representing the correlation between the container quantity and the service calling condition;
predicting the container quantity based on the full link monitoring data and the associated parameters to obtain a container quantity predicted value;
and adjusting the container quantity of the system according to the container quantity estimated value.
2. The method according to claim 1, wherein the predicting the number of containers based on the full link monitoring data and the associated parameters to obtain a container number estimated value comprises:
and inputting the full link monitoring data and the associated parameters into a prediction model to obtain a container quantity estimated value.
3. The method for adjusting the number of containers according to claim 2, wherein the predictive model is a time series model;
the training process of the time series model comprises the following steps:
acquiring historical full-link monitoring data meeting periodic requirements; wherein the historical full link monitoring data comprises historical container operating parameters and historical service parameters;
establishing historical association parameters according to the historical container operation parameters and the historical service parameters;
and training an initial time sequence model by using the historical full link monitoring data and the historical association parameters to obtain a time sequence model for predicting the number of containers.
4. The method according to claim 1, wherein the predicting the number of containers based on the full link monitoring data and the associated parameters to obtain a predicted value of the number of containers comprises:
inputting the full link monitoring data into a prediction model to obtain a container data initial prediction value;
determining a target container quantity adjusting proportion corresponding to the association parameters based on a corresponding mode of the set association parameters and the container quantity adjusting proportion;
and adjusting the initial estimated value of the container quantity according to the target container quantity adjusting proportion to obtain the estimated value of the container quantity.
5. The method for adjusting the number of containers according to claim 4, wherein the predictive model is a time series model;
the training process of the time series model comprises the following steps:
acquiring historical full-link monitoring data meeting periodic requirements; the historical full-link monitoring data comprises historical container operation parameters and historical service parameters;
and training an initial time sequence model by using the historical full link monitoring data to obtain a time sequence model for predicting the number of containers.
6. The method according to claim 1, wherein the acquiring full link monitoring data comprises:
collecting initial full link monitoring data;
cleaning the initial full-link monitoring data according to a set data cleaning mode to obtain full-link monitoring data; the data cleaning mode comprises abnormal data elimination, missing data completion and parameter normalization.
7. The method according to any one of claims 1 to 6, wherein the estimated value of the number of containers comprises estimated values of the number of containers corresponding to a plurality of time periods; the adjusting the number of containers of the system according to the estimated value of the number of containers comprises:
based on a target time period to which the current time of the system belongs, adjusting the container quantity of the system to a container quantity estimated value corresponding to the target time period; wherein the target time period is any one of the plurality of time periods.
8. The adjusting device for the number of containers is characterized by comprising an acquisition unit, a construction unit, a prediction unit and an adjusting unit;
the acquisition unit is used for acquiring full link monitoring data; wherein the full link monitoring data comprises container operation parameters and service parameters; the service parameters are used for representing service calling conditions;
the construction unit is used for constructing a correlation parameter according to the container operation parameter and the service parameter; the correlation parameters are used for representing the correlation between the container quantity and the service calling condition;
the prediction unit is used for predicting the container quantity based on the full link monitoring data and the associated parameters to obtain a container quantity estimated value;
and the adjusting unit is used for adjusting the container quantity of the system according to the container quantity estimated value.
9. An apparatus for adjusting the number of containers, comprising:
a memory for storing a computer program;
a processor for executing said computer program for carrying out the steps of the method for adjusting the number of containers according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for adjusting the number of containers according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211165647.0A CN115390995A (en) | 2022-09-23 | 2022-09-23 | Method, device, equipment and medium for adjusting number of containers |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211165647.0A CN115390995A (en) | 2022-09-23 | 2022-09-23 | Method, device, equipment and medium for adjusting number of containers |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115390995A true CN115390995A (en) | 2022-11-25 |
Family
ID=84128010
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211165647.0A Pending CN115390995A (en) | 2022-09-23 | 2022-09-23 | Method, device, equipment and medium for adjusting number of containers |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115390995A (en) |
-
2022
- 2022-09-23 CN CN202211165647.0A patent/CN115390995A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022267735A1 (en) | Service data processing method and apparatus, computer device, and storage medium | |
| CN111045814B (en) | Resource scheduling method and terminal equipment | |
| WO2019196427A1 (en) | Method and device for optimizing backup on the basis of supporting fault event constrained unit commitment | |
| CN113515382B (en) | Cloud resource allocation method and device, electronic equipment and storage medium | |
| CN109992866A (en) | Training method, device, readable medium and the electronic equipment of load forecasting model | |
| CN111340240A (en) | Method and device for realizing automatic machine learning | |
| CN114500339B (en) | Node bandwidth monitoring method and device, electronic equipment and storage medium | |
| CN109088747A (en) | The management method and device of resource in cloud computing system | |
| CN113268403A (en) | Time series analysis and prediction method, device, equipment and storage medium | |
| CN112365070A (en) | Power load prediction method, device, equipment and readable storage medium | |
| CN110096339B (en) | System load-based capacity expansion and contraction configuration recommendation system and method | |
| CN110532156A (en) | A kind of capacity prediction methods and device | |
| CN115471215B (en) | Business process processing method and device | |
| US20140122403A1 (en) | Loading prediction method and electronic device using the same | |
| US11636377B1 (en) | Artificial intelligence system incorporating automatic model updates based on change point detection using time series decomposing and clustering | |
| US11651271B1 (en) | Artificial intelligence system incorporating automatic model updates based on change point detection using likelihood ratios | |
| CN115390995A (en) | Method, device, equipment and medium for adjusting number of containers | |
| CN114816758B (en) | Resource allocation method and device | |
| CN116628231B (en) | Task visual release method and system based on big data platform | |
| CN110298742B (en) | Data processing method and device | |
| CN113064714A (en) | Method and device for processing task by using idle resource in research and development environment | |
| CN115858921A (en) | Model processing method, device, equipment and storage medium | |
| CN116780755A (en) | Cloud resource scheduling operation system | |
| CN116307159A (en) | Load prediction method and device, electronic equipment and storage medium | |
| CN115220872A (en) | Container adjusting method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |