CN116954880A - Resource allocation method and device, electronic equipment and storage medium - Google Patents

Resource allocation method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116954880A
CN116954880A CN202210854741.0A CN202210854741A CN116954880A CN 116954880 A CN116954880 A CN 116954880A CN 202210854741 A CN202210854741 A CN 202210854741A CN 116954880 A CN116954880 A CN 116954880A
Authority
CN
China
Prior art keywords
resource
target service
service container
determining
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210854741.0A
Other languages
Chinese (zh)
Inventor
种璟
张勇
王丽
赵博
谢嗣超
谭敏
秦璐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Chengdu ICT Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Chengdu ICT Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Chengdu ICT Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN202210854741.0A priority Critical patent/CN116954880A/en
Publication of CN116954880A publication Critical patent/CN116954880A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022Mechanisms to release resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5044Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application discloses a resource allocation method and device, electronic equipment and a storage medium, wherein the resource allocation method comprises the following steps: acquiring the resource usage amount change condition of a target service container in the current period; determining a prediction model matched with the change condition of the resource usage amount; determining the resource demand of the target service container in a target period through the prediction model; determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand. Therefore, the resource demand of the next time period is predicted based on the prediction model corresponding to the current resource quantity change condition, so that different expansion modes are flexibly selected for dynamically distributing resources, the influence time length of insufficient resources is reduced, more accurate and flexible resource expansion control is provided, and the usability of the system is improved.

Description

Resource allocation method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of network technologies, and in particular, to a method and apparatus for allocating resources, an electronic device, and a storage medium.
Background
The resource management schemes related to the related art tend to have poor flexibility for integrating and expanding resources, for example, the abnormality is often detected after occurrence of a shortage of resources, and then the resource expansion is performed. Therefore, higher time delay exists, the performance is greatly reduced, the resource integration and expansion mode is fixed and single, and different resource use conditions cannot be flexibly dealt with.
Disclosure of Invention
In view of the above, the present invention provides a resource allocation method and apparatus, an electronic device, and a storage medium.
The technical scheme of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a resource allocation method, including:
acquiring the resource usage amount change condition of a target service container in the current period;
determining a prediction model matched with the change condition of the resource usage amount;
determining the resource demand of the target service container in a target period through the prediction model;
determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand.
Further, the obtaining the resource usage change condition of the target service container in the current period includes:
acquiring the resource usage of a target service container at a plurality of preset moments according to a preset period;
and determining the resource usage change condition of the target service container in the current period based on the resource usage corresponding to a plurality of preset moments contained in the current period.
Further, the determining a prediction model matched with the resource usage change condition includes:
determining the resource usage rate corresponding to a plurality of preset moments contained in the current period according to the resource usage rate change condition;
A predictive model is determined that matches the rate of change of the resource usage.
Further, the determining a prediction model that matches the rate of change of the resource usage comprises:
if the resource usage rate between two adjacent preset moments is larger than a preset threshold, determining that the prediction model is a Gompertz (Gompertz) curve model;
and if the resource usage increase rate is smaller than or equal to a preset threshold value within a preset duration, determining that the prediction model is a moving average model.
Further, the determining, by the predictive model, the resource demand of the target service container in the target period includes:
if the prediction model is a Gompertz curve model, determining the resource demand of the target service container in a target period based on the target period and preset curve model parameters;
and if the prediction model is a moving average model, determining the resource demand of the target service container in the target period based on the resource usage amount corresponding to each preset moment in the current period and the duration corresponding to the current period.
Further, the determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand comprises:
And determining to allocate resources for the target service container through vertical expansion or horizontal expansion based on the resource demand and the weight value corresponding to the target service container.
Further, the determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand comprises:
if the resource demand is smaller than the resource availability of the host in which the target service container is located, allocating resources for the target service container by vertically expanding the target service container;
and if the resource demand is greater than or equal to the resource availability of the host in which the target service container is located, allocating resources for the target service container by horizontally expanding the host.
Further, the method further comprises:
comparing the target resource amount allocated for the target service container with the resource demand amount;
if the target resource amount is larger than the resource demand amount, recovering the resources allocated to the target service container based on the number of instance copies in the target service container;
and if the target resource amount is smaller than the resource demand amount, supplementing and distributing resources to the target service container based on the number of instance copies in the target service container.
Further, the reclaiming the resources allocated to the target service container based on the number of instance copies in the target service container includes:
recovering resources allocated to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient;
the supplementing and distributing resources to the target service container based on the number of instance copies in the target service container comprises the following steps:
and supplementing and distributing resources to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient.
In a second aspect, an embodiment of the present invention provides a resource allocation apparatus, including:
the acquisition unit is used for acquiring the resource usage amount change condition of the target service container in the current period;
the determining unit is used for determining a prediction model matched with the resource usage amount change condition;
a computing unit for determining the resource demand of the target service container in a target period through the prediction model;
an allocation unit for determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand.
Further, the acquiring unit is specifically configured to:
Acquiring the resource usage of a target service container at a plurality of preset moments according to a preset period;
and determining the resource usage change condition of the target service container in the current period based on the resource usage corresponding to a plurality of preset moments contained in the current period.
Further, the determining unit is specifically configured to:
determining the resource usage rate corresponding to a plurality of preset moments contained in the current period according to the resource usage rate change condition;
a predictive model is determined that matches the rate of change of the resource usage.
Further, the determining unit is specifically configured to:
if the resource usage rate between two adjacent preset moments is larger than a preset threshold, determining that the prediction model is a Gompertz curve model;
and if the resource usage increase rate is smaller than or equal to a preset threshold value within a preset duration, determining that the prediction model is a moving average model.
Further, the computing unit is specifically configured to:
if the prediction model is a Gompertz curve model, determining the resource demand of the target service container in a target period based on the target period and preset curve model parameters;
and if the prediction model is a moving average model, determining the resource demand of the target service container in the target period based on the resource usage amount corresponding to each preset moment in the current period and the duration corresponding to the current period.
Further, the distribution unit is specifically configured to:
and determining to allocate resources for the target service container through vertical expansion or horizontal expansion based on the resource demand and the weight value corresponding to the target service container.
Further, the distribution unit is specifically configured to:
if the resource demand is smaller than the resource availability of the host in which the target service container is located, allocating resources for the target service container by vertically expanding the target service container;
and if the resource demand is greater than or equal to the resource availability of the host in which the target service container is located, allocating resources for the target service container by horizontally expanding the host.
Further, the distribution unit is further configured to:
comparing the target resource amount allocated for the target service container with the resource demand amount;
if the target resource amount is larger than the resource demand amount, recovering the resources allocated to the target service container based on the number of instance copies in the target service container;
and if the target resource amount is smaller than the resource demand amount, supplementing and distributing resources to the target service container based on the number of instance copies in the target service container.
Further, the distribution unit is specifically configured to:
recovering resources allocated to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient;
and supplementing and distributing resources to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient.
In a third aspect, an embodiment of the present invention provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor;
the processor, when executing the computer program, performs the steps of the method of one or more of the preceding claims.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer-executable instructions; the computer-executable instructions, when executed by the processor, are capable of performing the method of one or more of the foregoing aspects.
The resource allocation method provided by the embodiment of the invention comprises the following steps: acquiring the resource usage amount change condition of a target service container in the current period; determining a prediction model matched with the change condition of the resource usage amount; determining the resource demand of the target service container in a target period through the prediction model; determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand. Therefore, a proper prediction model is flexibly selected based on the current resource quantity change condition, so that the resource demand quantity of the next period is predicted, the resource expansion control can be performed in advance, and the time length of system performance reduction caused by insufficient resources is reduced. On the basis, different expansion modes are flexibly selected to dynamically allocate resources, so that more accurate and flexible resource expansion control can be provided, and the availability of the system is improved.
Drawings
Fig. 1 is a flow chart of a resource allocation method according to a first embodiment of the present invention;
fig. 2 is a flow chart of a resource allocation method according to a second embodiment of the present invention;
fig. 3 is a flowchart illustrating a resource allocation method according to a third embodiment of the present invention;
fig. 4 is a flowchart illustrating a resource allocation method according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a resource allocation device according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a resource allocation system according to an embodiment of the present invention;
fig. 7 is a flowchart illustrating a resource allocation method according to a fifth embodiment of the present invention;
fig. 8 is a flowchart illustrating a resource allocation method according to a sixth embodiment of the present invention.
Detailed Description
The present invention will be further described in detail with reference to the accompanying drawings, for the purpose of making the objects, technical solutions and advantages of the present invention more apparent, and the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present invention.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.
In the following description, the terms "first", "second", "third" and the like are merely used to distinguish similar objects and do not represent a particular ordering of the objects, it being understood that the "first", "second", "third" may be interchanged with a particular order or sequence, as permitted, to enable embodiments of the invention described herein to be practiced otherwise than as illustrated or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.
As shown in fig. 1, an embodiment of the present invention provides a resource allocation method, where the method includes:
s110: acquiring the resource usage amount change condition of a target service container in the current period;
s120: determining a prediction model matched with the change condition of the resource usage amount;
s130: determining the resource demand of the target service container in a target period through the prediction model;
s140: determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand.
In the embodiment of the invention, the target service container can provide an operation environment for micro services and the like, wherein the target service container can be one or more service containers in a physical machine, namely a host machine, the target service container can be a virtual machine, and the host machine to which the virtual machine belongs can be considered as a node in the cluster.
Here, the resource usage amount in the target service container may include at least one of: central processing unit (central processing unit, CPU) utilization, memory usage, disk throughput, network throughput, application task scheduler load, and the like. The network throughput may include an intranet throughput and/or an extranet throughput, among others.
In one embodiment, obtaining the resource usage change condition of the target service container in the current period may include: and acquiring the resource usage change condition of the target resource of the target service container in the current period. The target resource may be one of a CPU, a memory, a disk, an application task scheduler, and the like.
In one embodiment, obtaining the resource usage change condition of the target service container in the current period may include: and determining the resource usage change condition of the current period based on the resource usage of the target service container and the weight corresponding to each resource.
For example, the weighted calculation can be performed based on the resource usage amounts of multiple resources of the target service container and the weight corresponding to each resource to obtain the resource usage amount at the current moment; and determining the resource usage change condition of the current period based on the resource usage of the current moment.
In one embodiment, the current period may be a preset period including the current time, for example, a preset period having the current time as a termination time or an intermediate time.
In one embodiment, the predictive model may include a Gompertz (Gompertz) curve model, a moving average model, or the like. The prediction model matched with the resource usage amount change condition may be a prediction model matched with at least one of a resource usage amount change trend, a resource usage amount change amount, a resource usage amount change rate, and the like.
For example, when the change trend of the resource usage in the current period is a growing trend and the change rate of the resource usage reaches a preset threshold, the matched prediction model is determined to be a Gompertz curve model.
In one embodiment, the resource demand of the target service container in the target period is determined through the prediction model, and the resource demand of the target service container in the target period can be determined through the prediction model and the change condition of the resource usage in the current period.
For example, the resource demand of the target service container in the target period is determined by the prediction model and the resource usage at the current time, or by the prediction model and the resource usage change amount and/or the resource usage change rate of the current period, or the like.
For example, the model parameters of the prediction model may be determined based on the resource usage amount at the current time, or the resource usage amount change amount and/or the resource usage amount change rate at the current time period, or the like; the resource demand of the target service container during the target period is determined based on the model parameters.
In one embodiment, determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand may include: and determining to allocate resources for the target service container through vertical expansion or horizontal expansion based on the resource demand and the resource availability of the host in which the target service container is located.
For example, when the resource demand is less than the resource availability, the resource is allocated to the target service container by vertical expansion.
And when the resource demand is greater than or equal to the resource availability, allocating resources to the target service container through horizontal expansion.
Therefore, a proper prediction model is flexibly selected based on the current resource quantity change condition, so that the resource demand quantity of the next period is predicted, the resource expansion control can be performed in advance, and the time length of system performance reduction caused by insufficient resources is reduced. On the basis, different expansion modes are flexibly selected to dynamically allocate resources, so that more accurate and flexible resource expansion control can be provided, and the availability of the system is improved.
In some embodiments, as shown in fig. 2, the step S110 may include:
s111: acquiring the resource usage of a target service container at a plurality of preset moments according to a preset period;
s112: and determining the resource usage change condition of the target service container in the current period based on the resource usage corresponding to a plurality of preset moments contained in the current period.
In the embodiment of the present disclosure, the predetermined period may be a preset acquisition period for acquiring the resource usage amount at a corresponding predetermined time in a fixed period, that is, a time interval between every two adjacent predetermined times may be a predetermined period.
In one embodiment, the resource usage of the target service container may also be collected according to a collection period, e.g., the collection period may be less than a predetermined period for obtaining the resource usage, or the collection period may be 0, i.e., the resource usage is collected in real time. The obtaining the resource usage of the target service container at a plurality of preset moments according to the preset period may include: and acquiring the resource usage amount of the target service container acquired at a plurality of preset moments according to the acquisition period according to the preset period.
In one embodiment, obtaining the resource usage of the target service container at a plurality of predetermined times according to the predetermined period may include: and according to the preset period, acquiring the resource usage of the target service container at a plurality of preset moments based on the resource usage of a plurality of resources of the target service container and the weight corresponding to each resource, and obtaining the resource usage at a plurality of preset moments.
In one embodiment, step S112 may include: determining the resource usage amount change condition between every two adjacent preset moments based on the resource usage amounts corresponding to the preset moments contained in the current time period; and determining the resource usage change condition of the target service container in the current period based on the resource usage change condition between every two adjacent preset moments.
For example, the resource usage amount change amount and/or the resource usage amount change rate between every two adjacent predetermined times are determined based on the resource usage amounts corresponding to the plurality of predetermined times included in the current period; and determining the resource usage amount change amount and/or the resource usage amount change rate of the target service container in the current period based on the average value of the resource usage amount change amount and/or the resource usage amount change rate between every two adjacent preset moments contained in the current period.
Illustratively, if the current period includes 3 predetermined times t1, t2 and t3, the resource usage amount change of the target service container in the current period may be an average value of the resource usage amount change between t1 and t2 and the resource usage amount change between t2 and t 3.
Thus, the periodic resource usage monitoring is performed according to the preset period, so that the granularity and the stability of resource monitoring and control can be improved, and the usability of the system can be improved.
In some embodiments, as shown in fig. 3, the step S120 may include:
s121: determining the resource usage rate corresponding to a plurality of preset moments contained in the current period according to the resource usage rate change condition;
s122: a predictive model is determined that matches the rate of change of the resource usage.
In embodiments of the present disclosure, the predictive model may include a Gompertz curve model, a moving average model, or the like. Different resource usage rates may correspond to different prediction models, e.g., a Gompertz curve model may be selected as the matched prediction model when the resource usage rate is above a predetermined threshold.
In one embodiment, the rate of change of the resource usage corresponding to the plurality of predetermined times included in the current period may be the rate of change of the resource usage at each predetermined time included in the current period compared to the previous predetermined time. For example, the rate of change of the resource usage may be the amount of change of the resource usage at each predetermined time as compared to the resource usage at the previous predetermined time divided by the length of the interval between the adjacent two predetermined times.
In some embodiments, the S122 may include:
if the resource usage rate between two adjacent preset moments is larger than a preset threshold, determining that the prediction model is a Gompertz curve model;
and if the resource usage increase rate is smaller than or equal to a preset threshold value within a preset duration, determining that the prediction model is a moving average model.
Here, the preset threshold may be a preset rate of increase in the amount of use of resources, such as an amount of increase in the amount of use of resources per unit time period, or an amount of increase in the ratio of use of resources per unit time period, or the like.
In one embodiment, the rate of increase in resource usage may be negative, i.e., a decrease in resource usage at a corresponding predetermined time as compared to a previous predetermined time is characterized.
In one embodiment, the preset duration may be a fixed value, or may be determined according to a predetermined period, for example, the preset duration is N predetermined periods, etc. Wherein N may be a positive integer greater than 0.
In one embodiment, the resource usage rate is less than or equal to the preset threshold value in the preset duration, and the resource usage rate corresponding to a plurality of preset moments in the preset duration may be less than or equal to the preset threshold value.
In one embodiment, the resource usage increase rate is less than or equal to the preset threshold value in the preset duration, and the number of the preset moments when the resource usage increase rate is less than or equal to the preset threshold value in the preset duration reaches the preset number or the duty ratio reaches the preset duty ratio.
In one embodiment, the rate of increase of the resource usage is less than or equal to the preset threshold for the preset duration, and may be less than or equal to the preset threshold for one or more predetermined periods within the preset duration.
Therefore, for the rapid change of the quantity or the rate of change of the resource usage possibly caused by sporadic resource jitter in the system, the influence caused by false alarm is reduced, and the resource expansion processing is performed for the continuous stable requirement in the preset time.
In some embodiments, the S130 may include:
if the prediction model is a Gompertz curve model, determining the resource demand of the target service container in a target period based on the target period and preset curve model parameters;
and if the prediction model is a moving average model, determining the resource demand of the target service container in the target period based on the resource usage amount corresponding to each preset moment in the current period and the duration corresponding to the current period.
In the embodiment of the disclosure, the increase rate of the resource usage is lower than or equal to the preset threshold, which indicates that the increase of the resource usage is stable or fast, and the average movement model is used as a prediction algorithm, so that the resource prediction in a stable or descending state can be more attached.
In one embodiment, a rate of increase in resource usage is greater than a preset threshold, indicating a rapid increase in resource usage, predicted using Gompertz curve model fitting, for example, byCalculating the resource demand y of the target period t t . Where K, a and b are model parameters of the Gompertz curve model.
In one embodiment, the moving average model may perform nonlinear least square fitting on the resource usage amount corresponding to each predetermined time included in the current period based on a gaussian newton algorithm, so as to obtain a curve growth trend. For example, the sum of the resource usage amounts corresponding to all the preset moments contained in the current period is divided by the duration corresponding to the current period to obtain the target timeResource demand y of segment t t
In some embodiments, as shown in fig. 4, the step S140 may include:
s141: and determining to allocate resources for the target service container through vertical expansion or horizontal expansion based on the resource demand and the weight value corresponding to the target service container.
In one embodiment, step S141 may include: determining a target resource amount allocated to the target service container based on the resource demand amount and a weight value corresponding to the target service container; determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the target resource amount.
In one embodiment, the weight value corresponding to the target service container may be a weight value corresponding to a node where the target service container is located, i.e. the host. For example, the weight value may be a load factor, which is used to label the load weight of each service container in the host. If the resource demand is the same, the higher the weight value of the target service container is, the higher the target resource amount allocated to the target service container is.
In one embodiment, the target resource amount may be a sum of the calculated resource amount and a preset redundancy amount based on the resource demand amount and a weight value corresponding to the target service container. The preset redundancy amount can be an additionally set redundancy resource amount of the target service container, so that the condition of insufficient resources caused by errors of the target resource amount is reduced, and the fault tolerance rate of resource allocation is improved.
For example, the preset amount of redundancy may be determined according to at least one of usage of the target service container in the host, usage of resources, and variation of usage of resources.
In one embodiment, the weight value may be a fixed value that is set in advance, or may be determined based on the usage of the target service container in the host. For example, based on the usage of the target service container relative to all service containers in the host.
Illustratively, the higher the usage of the target service container in the host, the higher the corresponding weight value may be.
In one embodiment, the weight value may also be determined by an unsupervised k-means clustering algorithm, for example, by performing classification dynamic labeling by the algorithm, to obtain a corresponding weight value.
Therefore, the resource allocation strategies of different service containers can be flexibly and dynamically set through the labeling and the setting of the weight values, so that the flexibility of resource control and the matching degree with the service containers are further improved.
In some embodiments, the S140 may include:
if the resource demand is smaller than the resource availability of the host in which the target service container is located, allocating resources for the target service container by vertically expanding the target service container;
and if the resource demand is greater than or equal to the resource availability of the host in which the target service container is located, allocating resources for the target service container by horizontally expanding the host.
In the embodiment of the disclosure, the vertical expansion of the target service container may be that in a host where the target service container is located, resources are allocated to the target service container based on resources of the host. For example, by invoking the free resources of other service containers to allocate to the target service container.
In one embodiment, the horizontal extension of the host may be to allocate resources to the target service container through the resources of the host to which the target service container belongs, that is, the other hosts in the system in which the physical machine is located. For example, other hosts that determine that the available amount of resources meets the resource demand or target amount of resources to be allocated, invoke allocation of the resources of that host to the target service container.
Thus, the vertical expansion is preferentially executed when the host resources are available, and the horizontal expansion is executed when the vertical expansion cannot meet the requirements, so that the hybrid scheduling strategy can provide efficient and fine-grained resource expansion control, and meanwhile, the high availability of the whole system is ensured.
In some embodiments, the method may further comprise:
comparing the target resource amount allocated for the target service container with the resource demand amount;
if the target resource amount is larger than the resource demand amount, recovering the resources allocated to the target service container based on the number of instance copies in the target service container;
And if the target resource amount is smaller than the resource demand amount, supplementing and distributing resources to the target service container based on the number of instance copies in the target service container.
In the embodiment of the disclosure, if the target resource amount is greater than the resource demand amount, which indicates that the amount of resources allocated to the target service container is excessive, part of the resources need to be recovered. And recycling resources based on the number of instance copies, wherein the resources can be recycled based on the number of instance copies in the target service container and a preset adjustment coefficient.
In some embodiments, the reclaiming the resources allocated for the target service container based on the number of instance copies in the target service container may include:
recovering resources allocated to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient;
the supplementing and distributing resources to the target service container based on the number of instance copies in the target service container comprises the following steps:
and supplementing and distributing resources to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient.
In the embodiment of the present disclosure, based on the number of instance copies in the target service container and a preset adjustment coefficient, the amount of reclaimed resources for reclaiming resources allocated to the target service container may be a difference value between the amount of resource demand and a value obtained by dividing the amount of resource usage of the target server at the current time or in the current period by the instance copy parameter. The instance copy parameter may be a product of the number of instance copies in the target service container and a preset adjustment coefficient.
For example, recRsc according to the formula r =req r -sum(usage r )/(tag m X p) determining reclaimed resourcesAmount RecRsc r Wherein r is a target resource, req r To the resource demand, sum (use r ) For the resource usage amount of the target server at the current moment or in the current period, tag m For the number of instance copies in the target service container, p is the adjustment factor.
The value range of the adjustment coefficient may be (0, 1), and a specific value may be preset.
In one embodiment, based on the number of instance copies in the target service container and a preset adjustment coefficient, resources are allocated to the target service container in a complementary manner, and the difference between the value obtained by dividing the resource usage of the target server at the current moment or in the current period by the instance copy parameter and the resource demand can be obtained. The instance copy parameter may be a product of the number of instance copies in the target service container and a preset adjustment coefficient.
For example, according to the formula RqrRSC r =sum(usage r )/(tag m ×p)-req r Determining the amount of the recovered resources, wherein r is a target resource, req r To the resource demand, sum (use r ) For the resource usage amount of the target server at the current moment or in the current period, tag m For the number of instance copies in the target service container, p is the adjustment factor.
The value range of the adjustment coefficient may be (0, 1), and a specific value may be preset.
In one embodiment, the method may further comprise: and if the target resource quantity is equal to the resource demand quantity, ending the current resource scheduling period. For example, ending the current resource scheduling period may be for stopping the current resource scheduling operation.
Therefore, based on recycling and supplementing resource allocation, the rationality of resource control can be further improved, the accuracy of resource use is improved, and the system availability difference caused by resource waste and resource deficiency is reduced.
As shown in fig. 5, an embodiment of the present invention provides a resource allocation apparatus, including:
an obtaining unit 10, configured to obtain a resource usage amount change condition of the target service container in the current period;
a determining unit 20, configured to determine a prediction model that matches the resource usage amount change condition;
a calculation unit 30 for determining a resource demand of the target service container in a target period by the prediction model;
an allocation unit 40 for determining allocation of resources to the target service container by vertical expansion or horizontal expansion based on the resource demand.
In some embodiments, the acquiring unit 10 is specifically configured to:
acquiring the resource usage of a target service container at a plurality of preset moments according to a preset period;
And determining the resource usage change condition of the target service container in the current period based on the resource usage corresponding to a plurality of preset moments contained in the current period.
In some embodiments, the determining unit 20 is specifically configured to:
determining the resource usage rate corresponding to a plurality of preset moments contained in the current period according to the resource usage rate change condition;
a predictive model is determined that matches the rate of change of the resource usage.
In some embodiments, the determining unit 20 is specifically configured to:
if the resource usage rate between two adjacent preset moments is larger than a preset threshold, determining that the prediction model is a Gompertz curve model;
and if the resource usage increase rate is smaller than or equal to a preset threshold value within a preset duration, determining that the prediction model is a moving average model.
In some embodiments, the computing unit 30 is specifically configured to:
if the prediction model is a Gompertz curve model, determining the resource demand of the target service container in a target period based on the target period and preset curve model parameters;
and if the prediction model is a moving average model, determining the resource demand of the target service container in the target period based on the resource usage amount corresponding to each preset moment in the current period and the duration corresponding to the current period.
In some embodiments, the distribution unit 40 is specifically configured to:
and determining to allocate resources for the target service container through vertical expansion or horizontal expansion based on the resource demand and the weight value corresponding to the target service container.
In some embodiments, the distribution unit 40 is specifically configured to:
if the resource demand is smaller than the resource availability of the host in which the target service container is located, allocating resources for the target service container by vertically expanding the target service container;
and if the resource demand is greater than or equal to the resource availability of the host in which the target service container is located, allocating resources for the target service container by horizontally expanding the host.
In some embodiments, the dispensing unit 40 is further configured to:
comparing the target resource amount allocated for the target service container with the resource demand amount;
if the target resource amount is larger than the resource demand amount, recovering the resources allocated to the target service container based on the number of instance copies in the target service container;
and if the target resource amount is smaller than the resource demand amount, supplementing and distributing resources to the target service container based on the number of instance copies in the target service container.
In some embodiments, the distribution unit 40 is specifically configured to:
recovering resources allocated to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient;
and supplementing and distributing resources to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient.
A specific example is provided below in connection with any of the above embodiments:
as shown in fig. 6, an embodiment of the present invention provides a machine learning cloud resource scheduling method for containerized micro-services, which is based on a containerized operation mode as a whole. As shown in fig. 7, an active cloud resource management model based on resource usage is used, including: the system comprises a performance detection monitoring module, a resource quantity prediction analysis module, a resource supply decision analysis module and a resource allocation and recovery module, wherein the resource allocation module comprises a vertical expansion module and a horizontal expansion module.
As shown in fig. 8, the specific steps include:
first, collecting performance detection data of the nodes. Periodically (custom setting according to requirement, 0 is real-time detection) monitoring and obtaining the resource usage node of the target resource r of the service container on the node (node) in the cluster useage_r (wherein node is node number, r is target resource item number for monitoring). The resources comprise one or more of multi-dimensional custom monitoring items such as CPU utilization rate, memory usage amount, IO throughput of a disk, IO throughput of a network (including an external network and an internal network), load amount of an application task scheduler and the like, real-time load information of a target node is recorded in a real-time monitoring mode, the weight of different monitoring items (the resource consumption value corresponding to the custom setting monitoring target item) is used for setting, and the total resource usage U of each current node is registered and marked usage =∑node useage_r
And secondly, predicting the number of resources. The resource quantity prediction module uses the data monitored, collected and processed by the node performance monitor as input. The system periodically performs resource monitoring and scheduling, and acquires the resource usage U at the mth time usage_m . The next data read time interval is delta t The m+1st time resource usage is U usage_(m+1) Taking the data monitored between two resource scheduling operations as a data point { U } usage_m ,U usage_(m+1) }. Next, resource demand prediction is performed, and a window, i.e., a current period, in which a plurality of data form one historical data point is required, e.g., the current period is expressed as time_windows_size。
The resource quantity predicting module predicts the quantity of various kinds of resources required in the next time period by using the historical data provided by the performance monitor, and the quantity of various kinds of resources possibly required in the next time period is used as input data of the resource supplying module. The method of combining Gompertz curve model with time series algorithm is selected (in order to ensure that predictive model algorithm needs to have higher qualification rate Qr as much as possible, and deviation D as low as possible) r At the same time, in order to guarantee performance as much as possible, it is necessary to predict the amount of redundant resources θ higher than actually required).
The time sequence algorithm is characterized in that an actual value steadily or rapidly drops, and an average movement model is used as a prediction algorithm; when the actual value rises rapidly, a Gompertz curve model fitting is used for prediction, wherein K, a and b are parameters of the Gompertz curve model, historical data are input and summarized, nonlinear least square fitting is carried out on the latest monitored historical data points contained in the time_windows_size through a Gaussian Newton algorithm, a curve growth trend is obtained, and T (data sensitivity, namely, growth speed) and N (monitoring granularity, namely, monitoring period) are used for obtaining the curve growth trend.
And thirdly, resource supply decision analysis. Calculating the amount of resources MisRsc that the service container needs to additionally allocate m :MisRsc m =sum(usage r )/tag m -req r Wherein sum (use r ) Representing the total usage of the service container resources r in the cluster, req r Indicating the required allocation amount of the resource r, tag m Representing the number of copies of container instance m of the microservice.
The second step is mainly used for predicting the quantity of various resources required by the program in a future period of time to sense the future demand increase in advance when the flow suddenly increases, so that the cloud platform is prevented from violating the performance target of the program demand (the average response time of the program cannot be consistent with the agreed threshold value). When encounteringWhen the usage amount is reduced, the usage amount is also predicted to be reduced in advance, so that the whole model is guaranteed to realize high cost benefit, the actual situation can encounter some sporadic resource jitters, and in order to reduce the influence caused by false alarm, resource alarm pressing is used: the resource demand quota is in a stable period corresponding to the sensitivity M (M can be set in a self-defining way, namely the sensitivity), and the resource expansion processing is carried out after the resource demand quota is continuously stabilized. Furthermore, due to the need for redundancy of the dynamic adjustment system, the summary is:wherein Sigma allocated MisRsc For all resource usage of the target service container, +.>Is the amount of redundancy.
And fourthly, resource allocation and recovery. And on the basis of the third step, scheduling, distributing and recycling the resources. The method comprises the steps of carrying out load balancing setting on nodes in a cluster, carrying out self-classification dynamic labeling on the nodes by using a load factor (sum of resource demand numbers) as an example weight by using a load balancer, carrying out load distribution according to the weight, and obtaining a comprehensive weight value of the nodes by using an unsupervised k-means clustering algorithm.
Fifth, if the amount of resources allocated by the service is greater than the amount of resources required, i.e., misRsc m <0, then, a resource reclamation stage is entered. At this time, the amount of resources recovered from each instance is: recRsc r =req r -sum(usage r )/(tag m X p), wherein 0<p<And 1 is an adjustment coefficient set by an operation and maintenance engineer.
Sixth, if the amount of resources allocated by the service is less than the amount of resources required, i.e., misRsc m >0, then, a resource request phase is entered, at which the amount of resources that need to be increased from each instance is: rqrRsc r =sum(usage r )/(tag m ×p)-req r Wherein 0 is<p<And 1 is an adjustment coefficient set by an operation and maintenance engineer.
Seventh step, if clothingThe amount of resources allocated by the service is equal to the amount of resources required, i.e. MisRsc m =0, then the scheduling period ends.
Eighth, when entering the resource request stage, checking whether the available resources of the current host meet the requirement of carrying out resource expansion on the current host; if the available resource amount of the host is larger than the required increased resource amount, performing vertical expansion of the service container; if the amount of available resources of the host is less than the amount of resources that need to be increased, then a service container hierarchical extension is performed.
The method comprises the steps of adding the attribute of a related monitoring item (enumerating effective influence factors as much as possible according to actual conditions) into a scheduling weight association list, performing comprehensive investigation, dynamically monitoring multi-azimuth data, associating the level of influence with the attribute factors in the comprehensive list as much as possible, performing auxiliary setting of different weight values, further marking and identifying the priority of the monitoring item and the judging item more effectively, accurately judging whether the threshold condition for triggering elasticity is reached (effectively eliminating system false alarm or response delay caused by instantaneous jitter of certain monitoring resource items), and executing the elastic expansion function of the system through a self-contained inhibition function.
In addition, the action of instantiating the new copy can be triggered only when the resource requirement provided by the node to the micro-service is greater than the minimum threshold set by the operation and maintenance of the system, so that the performance jitter of the instance cannot be generated due to the fact that the resource allocation is too small; in addition, the system operation and maintenance needs to set a reasonable telescopic interval threshold to avoid frequent system telescopic operation and maintenance to avoid jitter.
The embodiment of the application also provides electronic equipment, which comprises: a processor and a memory for storing a computer program capable of running on the processor, which when run performs the steps of the method of one or more of the preceding claims.
The embodiment of the application also provides a computer readable storage medium, which stores computer executable instructions that, when executed by a processor, can implement the method according to one or more of the foregoing technical solutions.
The computer storage medium provided in this embodiment may be a non-transitory storage medium.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described as separate units may or may not be physically separate, and units displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present invention may be integrated in one processing module, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.
In some cases, the two technical features do not conflict, and a new method technical scheme can be combined.
In some cases, the above two technical features may be combined into a new device technical scheme without any conflict.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk or an optical disk, or the like, which can store program codes.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (12)

1. A method of resource allocation, the method comprising:
acquiring the resource usage amount change condition of a target service container in the current period;
determining a prediction model matched with the change condition of the resource usage amount;
determining the resource demand of the target service container in a target period through the prediction model;
determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand.
2. The method according to claim 1, wherein the obtaining the resource usage change condition of the target service container in the current period includes:
acquiring the resource usage of a target service container at a plurality of preset moments according to a preset period;
and determining the resource usage change condition of the target service container in the current period based on the resource usage corresponding to a plurality of preset moments contained in the current period.
3. The method of claim 2, wherein said determining a predictive model that matches the resource usage variation comprises:
determining the resource usage rate corresponding to a plurality of preset moments contained in the current period according to the resource usage rate change condition;
a predictive model is determined that matches the rate of change of the resource usage.
4. A method according to claim 3, wherein said determining a predictive model that matches said rate of change of resource usage comprises:
if the resource usage rate between two adjacent preset moments is larger than a preset threshold, determining that the prediction model is a Gompertz curve model;
and if the resource usage increase rate is smaller than or equal to a preset threshold value within a preset duration, determining that the prediction model is a moving average model.
5. The method of claim 4, wherein said determining, by said predictive model, a resource demand of said target service container over a target period of time comprises:
if the prediction model is a Gompertz curve model, determining the resource demand of the target service container in a target period based on the target period and preset curve model parameters;
And if the prediction model is a moving average model, determining the resource demand of the target service container in the target period based on the resource usage amount corresponding to each preset moment in the current period and the duration corresponding to the current period.
6. The method of claim 1, wherein the determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand comprises:
and determining to allocate resources for the target service container through vertical expansion or horizontal expansion based on the resource demand and the weight value corresponding to the target service container.
7. The method of claim 1, wherein the determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand comprises:
if the resource demand is smaller than the resource availability of the host in which the target service container is located, allocating resources for the target service container by vertically expanding the target service container;
and if the resource demand is greater than or equal to the resource availability of the host in which the target service container is located, allocating resources for the target service container by horizontally expanding the host.
8. The method according to claim 1, wherein the method further comprises:
comparing the target resource amount allocated for the target service container with the resource demand amount;
if the target resource amount is larger than the resource demand amount, recovering the resources allocated to the target service container based on the number of instance copies in the target service container;
and if the target resource amount is smaller than the resource demand amount, supplementing and distributing resources to the target service container based on the number of instance copies in the target service container.
9. The method of claim 1, wherein the reclaiming the resources allocated for the target service container based on the number of instance copies in the target service container comprises:
recovering resources allocated to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient;
the supplementing and distributing resources to the target service container based on the number of instance copies in the target service container comprises the following steps:
and supplementing and distributing resources to the target service container based on the number of instance copies in the target service container and a preset adjustment coefficient.
10. A resource allocation apparatus, the apparatus comprising:
the acquisition unit is used for acquiring the resource usage amount change condition of the target service container in the current period;
the determining unit is used for determining a prediction model matched with the resource usage amount change condition;
a computing unit for determining the resource demand of the target service container in a target period through the prediction model;
an allocation unit for determining to allocate resources to the target service container by vertical expansion or horizontal expansion based on the resource demand.
11. An electronic device, the electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor; wherein,,
the processor, when executing the computer program, performs the steps of the resource allocation method of any of claims 1 to 9.
12. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions; the computer executable instructions, when executed by a processor, are capable of implementing the resource allocation method according to any of claims 1 to 9.
CN202210854741.0A 2022-07-18 2022-07-18 Resource allocation method and device, electronic equipment and storage medium Pending CN116954880A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210854741.0A CN116954880A (en) 2022-07-18 2022-07-18 Resource allocation method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210854741.0A CN116954880A (en) 2022-07-18 2022-07-18 Resource allocation method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116954880A true CN116954880A (en) 2023-10-27

Family

ID=88441524

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210854741.0A Pending CN116954880A (en) 2022-07-18 2022-07-18 Resource allocation method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116954880A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117828389A (en) * 2024-03-05 2024-04-05 深圳市互盟科技股份有限公司 Distributed system monitoring method based on intelligent operation and maintenance

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117828389A (en) * 2024-03-05 2024-04-05 深圳市互盟科技股份有限公司 Distributed system monitoring method based on intelligent operation and maintenance
CN117828389B (en) * 2024-03-05 2024-06-04 深圳市互盟科技股份有限公司 Distributed system monitoring method based on intelligent operation and maintenance

Similar Documents

Publication Publication Date Title
EP3847549B1 (en) Minimizing impact of migrating virtual services
US11966820B2 (en) Utilizing machine learning models with a centralized repository of log data to predict events and generate alerts and recommendations
US10915369B2 (en) Reward-based admission controller for resource requests in the cloud
US11726836B2 (en) Predicting expansion failures and defragmenting cluster resources
CN106471473B (en) Mechanism for controlling server over-allocation in a data center
CN112799817A (en) Micro-service resource scheduling system and method
US11972301B2 (en) Allocating computing resources for deferrable virtual machines
US20150113539A1 (en) Method for executing processes on a worker machine of a distributed computing system and a distributed computing system
US20220300345A1 (en) Entity and Method Performed therein for Handling Computational Resources
US11398957B2 (en) Systems and methods for predicting failure of a network device
CN116954880A (en) Resource allocation method and device, electronic equipment and storage medium
Gupta et al. A supervised deep learning framework for proactive anomaly detection in cloud workloads
Buchaca et al. Proactive container auto-scaling for cloud native machine learning services
WO2020206699A1 (en) Predicting virtual machine allocation failures on server node clusters
KR20230069490A (en) Method for applying workload prediction model by operation service in micro data center
CN116560794A (en) Exception handling method and device for virtual machine, medium and computer equipment
US11960904B2 (en) Utilizing machine learning models to predict system events based on time series data generated by a system
CN114296872A (en) Scheduling method and device for container cluster management system
CN114201363A (en) System protection method, device, equipment and storage medium
CN114676002A (en) PHM technology-based system operation and maintenance method and device
Mustafa et al. Evolving stream classification using change detection
CN112835951A (en) Data prediction method, data prediction device, computer equipment and storage medium
US11991037B2 (en) Systems and methods for reducing a quantity of false positives associated with rule-based alarms
US20230237215A1 (en) Contingency forecasting system
Wu et al. A Novel Container Workload Prediction Method Based on Automatic Classification and Attention-based LSTM

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination