CN114866563A - Capacity expansion method, device, system and storage medium - Google Patents
Capacity expansion method, device, system and storage medium Download PDFInfo
- Publication number
- CN114866563A CN114866563A CN202210680103.1A CN202210680103A CN114866563A CN 114866563 A CN114866563 A CN 114866563A CN 202210680103 A CN202210680103 A CN 202210680103A CN 114866563 A CN114866563 A CN 114866563A
- Authority
- CN
- China
- Prior art keywords
- capacity expansion
- index data
- capacity
- resource
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000015654 memory Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 11
- 238000013135 deep learning Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 239000003102 growth factor Substances 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000008602 contraction Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1044—Group management mechanisms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1031—Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Theoretical Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a capacity expansion method, a capacity expansion device, a capacity expansion system and a storage medium, and relates to the field of cloud computing. The capacity expansion method comprises the following steps: acquiring current index data and historical index data; predicting index data of the future preemption time by using the current index data and the historical index data; determining the total capacity expansion according to the predicted index data of the preemption time; and expanding the capacity according to the total capacity expansion amount. The embodiment of the invention can guarantee the resource requirement of important business in a resource shortage scene in a resource preemption mode by predicting the capacity expansion of the future preemption time and expanding the capacity accordingly, thereby improving the resource utilization efficiency and the capacity expansion effectiveness.
Description
Technical Field
The invention relates to the field of cloud computing, in particular to a capacity expansion method, device, system and storage medium.
Background
In order to improve the utilization rate of resources and guarantee the service quality of services, two dynamic resource expansion and contraction methods, namely HPA (horizontal automatic expansion of containers) and VPA (Vertical automatic expansion of containers), are provided for the container cluster, and the number of the containers and the size of the resources are adjusted in real time according to the use condition of the resources. The related art provides a specific scheme as follows: 1) a user declares an expansion object, a trigger mechanism (index threshold) and resource limitation; 2) the HPA/VPA Controller receives the user declaration; 3) timing to index database (metric server) to inquire the expansion index of the statement; 4) if the index satisfies the threshold range stated by the user, a copy Controller (short for: RC) the amount of pod or resource (HPA and VPA, respectively) to be expanded; 5) and realizing the capacity expansion of resources.
Disclosure of Invention
After the analysis, the inventor finds that the edge computing node has the characteristics of limited resources and smaller scale. The capacity expansion mechanism of the container resource in the related art has certain limitations due to resource limitation in the scene. The use of the edge resources generally has the periodic characteristics of tidal effect and the like, and the resource construction and maintenance cost of the edge scene and the limited edge resources are considered, so that the resource construction cannot be carried out according to the peak condition of the resource hot spot time in the edge scene. Therefore, the resource load in the hot spot time period is high, and the service requirement cannot be met. Further, the scaling schemes provided by the related art have the following problems:
1) for the service with higher service quality requirement, the capacity expansion mechanism is triggered later, so that under the condition of resource shortage, insufficient resources are not available to realize effective capacity expansion through an HPA and VPA mechanism;
2) for the above services, if resource allocation is performed according to the maximum peak resource demand, the resource utilization efficiency in the non-hotspot time period will be low.
The embodiment of the invention aims to solve the technical problem that: how to improve the utilization efficiency of resources and improve the effectiveness of capacity expansion.
According to a first aspect of some embodiments of the present invention, there is provided a capacity expansion method, including: acquiring current index data and historical index data; predicting index data of the future preemption time by using the current index data and the historical index data; determining the total capacity expansion according to the predicted index data of the preemption time; and expanding the capacity according to the total capacity expansion amount.
In some embodiments, determining the total capacity expansion amount based on the predicted preemption time indicator data comprises: determining a resource increment corresponding to the preemption time according to the predicted index data of the preemption time; determining a capacity expansion coefficient according to the resource increment; predicting the capacity expansion probability of the preemption time by using the current index data, the historical index data and the historical capacity expansion event; and determining the total capacity expansion amount according to the resource increment and the capacity expansion coefficient corresponding to the preemption time and the predicted capacity expansion probability of the preemption time.
In some embodiments, the metrics data includes at least one of a user visit amount, a resource usage amount of the traffic, or a cluster resource remaining amount.
In some embodiments, determining the resource increment corresponding to the preemption time comprises: determining a predicted user growth coefficient according to the historical user access amount and the user access amount of the preemption time; determining a predicted service growth coefficient according to the historical service resource usage and the service resource usage of the seizing time; and determining the resource increment corresponding to the preemption time according to the maximum value of the user growth coefficient and the service resource usage amount of the preemption time.
In some embodiments, determining the capacity expansion coefficient based on the resource increment includes: and determining the capacity expansion coefficient according to the ratio of the resource increment to the predicted cluster resource residual quantity.
In some embodiments, determining the capacity expansion coefficient based on a ratio of the resource increment to the predicted remaining amount of the cluster resource comprises: determining the capacity expansion coefficient as 0 under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is smaller than a preset lower limit; or under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is between a preset lower limit and a preset upper limit, determining the capacity expansion coefficient as the ratio; or, under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is greater than a preset upper limit, determining the capacity expansion coefficient as 1.
In some embodiments, predicting a capacity expansion probability of a future preemption time using the current index data, the historical index data, and the historical capacity expansion event comprises: and processing the current index data, the historical index data and the historical capacity expansion event by using a deep learning network to obtain the capacity expansion probability of the future preemption time.
In some embodiments, determining the total capacity expansion amount according to the resource increment corresponding to the preemption time, the capacity expansion coefficient, and the predicted capacity expansion probability of the preemption time includes: and determining the product of the resource increment corresponding to the preemption time, the expansion coefficient and the predicted expansion probability of the preemption time as the total expansion capacity.
In some embodiments, using the current metric data and the historical metric data, the metric data that predicts future preemption times comprises: and processing the current index data and the historical index data by using a deep learning network to obtain the index data of the future preemption time.
In some embodiments, the current index data and the historical index data are acquired when a time interval between the current time and the last acquisition of the index data satisfies a preset time interval.
In some embodiments, the preemption time and time interval are user configured in a YAML file.
In some embodiments, the containers of the edge computing nodes are expanded based on the total expansion capacity.
According to a second aspect of some embodiments of the present invention, there is provided a capacity expansion apparatus, including: the acquisition module is configured to acquire current index data and historical index data; the prediction module is configured to predict index data of the future preemption time by using the current index data and the historical index data; a determining module configured to determine a total capacity according to the predicted indicator data of the preemption time; and the capacity expansion module is configured to expand the capacity according to the total capacity expansion amount.
According to a third aspect of some embodiments of the present invention, there is provided a capacity expansion system, including: the aforementioned capacity expansion means; a metrics server configured to query the current metric data; and an index database configured to query historical index data.
In some embodiments, the capacity expansion system further comprises: and the event server is configured to query the historical capacity expansion event.
In some embodiments, the capacity expansion system further comprises: and the capacity expansion device is used for expanding the capacity of the edge computing node.
According to a fourth aspect of some embodiments of the present invention, there is provided a capacity expansion device, including: a memory; and a processor coupled to the memory, the processor configured to perform any of the foregoing expansion methods based on instructions stored in the memory.
According to a fifth aspect of some embodiments of the present invention, there is provided a computer-readable storage medium on which a computer program is stored, wherein the program, when executed by a processor, implements any one of the capacity expansion methods described above.
Some embodiments of the above invention have the following advantages or benefits. The embodiment of the invention can guarantee the resource requirement of important business in a resource shortage scene in a resource preemption mode by predicting the capacity expansion of the future preemption time and expanding the capacity accordingly, thereby improving the resource utilization efficiency and the capacity expansion effectiveness.
Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 illustrates a flow diagram of a capacity expansion method according to some embodiments of the invention.
FIG. 2 is a flow diagram illustrating a method of determining a total capacity expansion amount according to some embodiments of the invention.
FIG. 3 is a flowchart illustrating a capacity expansion method according to other embodiments of the invention.
FIG. 4 illustrates a schematic diagram of a containment apparatus according to some embodiments of the invention.
FIG. 5 illustrates a schematic diagram of a capacity expansion system according to some embodiments of the invention.
Fig. 6 shows a schematic construction of a capacity enlarging device according to further embodiments of the invention.
FIG. 7 is a schematic diagram of a capacity expansion device according to still other embodiments of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
FIG. 1 illustrates a flow diagram of a capacity expansion method according to some embodiments of the invention. As shown in fig. 1, the capacity expansion method of this embodiment includes steps S102 to S108.
In step S102, the current index data and the historical index data are acquired. The current index data and the historical index data have the same meaning of the index, but the corresponding time is different.
In some embodiments, the metrics data includes at least one of a user visit amount, a resource usage amount of the traffic, or a cluster resource remaining amount. The user access amount is expressed, for example, in the number of user requests. The service resource usage represents the service resource usage, and the cluster resource surplus represents the overall resource usage.
In some embodiments, current metric data is read from the metric server and historical metric data is read from the metric database.
In some embodiments, the metric data may be acquired periodically. For example, by setting a time interval in advance, the acquisition process is triggered every preset time, and the method of the embodiment is executed. The preset time interval may be configured by the user, for example, in a YAML (a markup language) file.
In step S104, index data of a future preemption time is predicted using the current index data and the historical index data. Preemption time is the time served by a resource preemption behavior. Through the capacity expansion operation of the embodiment of the invention, more sufficient resources can be provided to meet the resource requirement when the preemption time comes.
In some embodiments, the current index data, the historical index data, and the historical capacity expansion event are processed by using a deep learning network, and the capacity expansion probability of the preemption time is obtained. Examples of deep learning networks include Long short-term memory (LSTM) networks, morphers (transformers), and the like. The historical capacity expansion event refers to an event corresponding to an occurred capacity expansion operation, and the historical capacity expansion event may include information such as occurrence time, an operation object, and an operation type. The current index data and the historical index data have time sequence characteristics, and the LSTM and the Transformer are also good at processing the data, so that the index data of the future preemption time can be predicted more accurately.
In some embodiments, the preemption time may be configured in a YAML file.
In step S106, the total capacity expansion amount is determined according to the index data of the predicted preemption time. Through the index data of the preemption time, the demand quantity of the corresponding resource during the preemption time can be evaluated, and through comparing the resource consumption situation at the moment, whether the capacity expansion is needed or not and the size of the total capacity expansion can be estimated.
In step S108, the capacity is expanded according to the total capacity expansion amount.
According to the embodiment, the capacity expansion of the future preemption time is predicted, and the capacity expansion is carried out according to the capacity expansion, so that the resource requirement of the important service can be guaranteed in a resource shortage scene in a resource preemption mode, and the efficiency of resource utilization is improved and the effectiveness of capacity expansion is improved.
An embodiment of determining the total capacity expansion amount based on the index data of the predicted preemption time is described below with reference to fig. 2.
FIG. 2 is a flow diagram illustrating a method of determining a total amount of capacity expansion according to some embodiments of the invention. As shown in fig. 2, the method of this embodiment includes steps S202 to S208.
In step S202, a resource increment corresponding to the preemption time is determined according to the predicted index data of the preemption time.
In some embodiments, a predicted user growth factor is determined based on historical user access and user access to preempt time; determining a predicted service growth coefficient according to the historical service resource usage and the service resource usage of the seizing time; and determining the resource increment corresponding to the preemption time according to the maximum value of the user growth coefficient and the service resource usage amount of the preemption time.
Let the current user access amount be u t Resource usage of current serviceDosage resS t The current cluster resource residual amount is REST t (ii) a Let the preemption time in the future be t 0 A time of day having a user access amount of
Traffic resource usage
Current cluster resource remaining amount
User growth factor, for example, in
Indicating the traffic growth factor, e.g. in
And (4) showing. The resource increment may be determined using equation (1), for example.
The determining mode considers the change condition of the user and the change condition of the service, and determines the resource increment by taking the more increased of the user and the service as a reference, so that the actual resource requirement can be better met.
In step S204, a capacity expansion coefficient is determined according to the resource increment.
In some embodiments, the capacity expansion factor is determined based on a ratio of the resource increment to the predicted remaining amount of cluster resources. The ratio of the resource increment to the remaining amount of cluster resources reflects the necessary degree of capacity expansion.
In some embodiments, when the ratio of the resource increment to the predicted cluster resource remaining amount is smaller than a preset lower limit, determining the capacity expansion coefficient to be 0; or under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is between a preset lower limit and a preset upper limit, determining the capacity expansion coefficient as the ratio; or, under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is greater than a preset upper limit, determining the capacity expansion coefficient as 1.
For example, first, a capacity expansion willingness coefficient δ is calculated by equation (2), which represents the degree of difference between the predicted resource increment and the current cluster resource remaining amount, thereby reflecting the necessary degree of capacity expansion.
And then, determining the expansion coefficient according to a piecewise function relation (3) between the expansion willingness coefficient delta and the expansion coefficient, wherein alpha represents a preset lower limit and beta represents a preset upper limit. The values of the two can be determined by a plurality of experiments or manually.
When delta < alpha indicates that the residual resources can sufficiently meet the requirement of resource increment, capacity expansion is not performed; when alpha is more than or equal to delta and less than or equal to beta represents that the residual resources have certain risks when meeting the requirement of the resource increment, capacity expansion is properly carried out according to delta; when delta > beta indicates that the residual resources are limited, capacity expansion is performed according to resource increment.
In step S206, the current index data, the historical index data, and the historical capacity expansion event are used to predict the capacity expansion probability of the preemption time.
In some embodiments, the current index data, the historical index data and the historical expansion event are processed by using a deep learning network, so that the expansion probability of the future preemption time is obtained. For example, the current index data, the historical index data and the historical capacity expansion event are input into the deep learning network for prediction.
In step S208, the total capacity expansion amount is determined according to the resource increment corresponding to the preemption time, the capacity expansion coefficient, and the predicted capacity expansion probability of the preemption time.
In some embodiments, the total expansion capacity is in a positive correlation with each of the resource increment corresponding to the preemption time, the expansion coefficient, and the predicted expansion probability of the preemption time.
In some embodiments, the product of the resource increment corresponding to the preemption time, the capacity expansion coefficient, and the predicted capacity expansion probability of the preemption time is determined as the total capacity expansion. For example, the total expansion capacity C is calculated using equation (4).
C＝incre×γ×p (4)
If C is 0, then no expansion is required; and if C is greater than 0, carrying out capacity expansion according to the value of C.
In the related art, when performing capacity expansion prediction, pre-expansion is generally performed only according to a comparison between a prediction situation of a traffic (e.g., user access volume) and a resource (e.g., cpu, memory) carried by a container itself and an actual value, and an overall resource load of a cluster is not considered, and a capacity expansion event itself is not predicted. According to the embodiment, whether the phenomenon of insufficient resources occurs after the set time or not and the quantity of the resources needing to be preempted in advance are predicted according to the predicted user access quantity and the service resource usage quantity cluster overall load, so that key services can be guaranteed, and effective capacity expansion can be realized under the condition of hot spot time or resource shortage.
In some embodiments, the prediction and expansion may be performed periodically. And under the condition that the time interval between the current moment and the last time of acquiring the index data meets the preset time interval, acquiring the current index data and historical index data.
FIG. 3 is a flowchart illustrating a capacity expansion method according to other embodiments of the invention. As shown in fig. 3, the capacity expansion method of this embodiment includes steps S302 to S310.
In step S302, the capacity expansion configuration is read, where the capacity expansion configuration includes a preset time interval and a preemption time.
In step S304, it is determined whether the current time meets a preset time interval requirement, that is, whether the time interval from the last time of acquiring the index data is a preset time interval. If yes, go to step S306; otherwise, waiting is carried out.
In addition, the process of reading the index and predicting can be triggered once every preset time interval.
In step S306, the current index data and the historical index data are read from the index database, and the historical capacity expansion event is read from the event database.
In step S308, the total capacity expansion amount is determined according to the data read in step S306. If the capacity expansion amount is greater than 0, executing step S310; otherwise, waiting is carried out.
In step S310, the capacity is expanded according to the total capacity expansion amount.
The embodiment obtains and predicts the data at intervals according to the configuration of the user, and expands the capacity when necessary, so that compared with a real-time capacity expansion mode in an HPA and VPA mechanism of the related art, frequent judgment and other processing are not needed, and the computing resources are saved.
FIG. 4 illustrates a schematic diagram of a containment apparatus according to some embodiments of the invention. As shown in fig. 4, the capacity expansion device 400 of this embodiment includes: an obtaining module 4100 configured to obtain current index data and historical index data; a prediction module 4200 configured to predict future preemption-time metric data using current metric data and historical metric data; a determining module 4300 configured to determine a total capacity according to the index data of the predicted preemption time; the capacity expansion module 4400 is configured to expand the capacity according to the total capacity expansion amount.
In some embodiments, the determining module 4300 is further configured to determine, according to the index data of the predicted preemption time, a resource increment corresponding to the preemption time; determining a capacity expansion coefficient according to the resource increment; predicting the capacity expansion probability of the preemption time by using the current index data, the historical index data and the historical capacity expansion event; and determining the total capacity expansion amount according to the resource increment and the capacity expansion coefficient corresponding to the preemption time and the predicted capacity expansion probability of the preemption time.
In some embodiments, the metrics data includes at least one of a user visit amount, a resource usage amount of the traffic, or a cluster resource remaining amount.
In some embodiments, the determining module 4300 is further configured to determine a predicted user growth factor based on the historical user visitation amounts and the user visitation amounts for the preemption time; determining a predicted service growth coefficient according to the historical service resource usage and the service resource usage of the seizing time; and determining the resource increment corresponding to the preemption time according to the maximum value of the user growth coefficient and the service resource usage amount of the preemption time.
In some embodiments, the determining module 4300 is further configured to determine a capacity expansion coefficient according to a ratio of the resource increment and the predicted remaining amount of the cluster resource.
In some embodiments, the determining module 4300 is further configured to determine the capacity expansion coefficient to be 0 if a ratio of the resource increment to the predicted remaining cluster resource amount is smaller than a preset lower limit; or under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is between a preset lower limit and a preset upper limit, determining the capacity expansion coefficient as the ratio; or, under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is greater than a preset upper limit, determining the capacity expansion coefficient as 1.
In some embodiments, the determining module 4300 is further configured to process the current indicator data, the historical indicator data, and the historical capacity expansion event using a deep learning network to obtain a capacity expansion probability of the future preemption time.
In some embodiments, the determining module 4300 is further configured to determine the product of the resource increment corresponding to the preemption time, the capacity expansion coefficient and the predicted capacity expansion probability of the preemption time as the total capacity expansion.
In some embodiments, the prediction module 4200 is further configured to process the current metric data and the historical metric data using a deep learning network to obtain future time-to-preempt metric data.
In some embodiments, the obtaining module 4100 is further configured to obtain the current index data and the historical index data if a time interval between the current time and the last time of obtaining the index data satisfies a preset time interval.
In some embodiments, the preemption time and time interval are user configured in a YAML file.
In some embodiments, the expansion module 4400 is further configured to expand the containers of the edge computing nodes according to the total expansion amount.
FIG. 5 illustrates a schematic diagram of a capacity expansion system according to some embodiments of the invention. As shown in fig. 5, the capacity expansion system 50 of this embodiment includes a capacity expansion device 510, and the specific implementation thereof may refer to the capacity expansion device 400; a metrics server 520 configured to query the current metric data; and an index database 530 configured to query historical index data.
In some embodiments, the capacity expansion system 50 further includes: and an event server 540 configured to query the historical capacity expansion event.
In some embodiments, the capacity expansion system 50 further includes: and an edge computing node 550, wherein the capacity expansion device expands the capacity of the edge computing node.
Fig. 6 shows a schematic configuration of a capacity expansion means according to further embodiments of the invention. As shown in fig. 6, the capacity expansion device 60 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610, wherein the processor 620 is configured to execute the capacity expansion method in any of the embodiments based on instructions stored in the memory 610.
FIG. 7 is a schematic diagram of a flash memory according to still other embodiments of the invention. As shown in fig. 7, the capacity expansion device 70 of this embodiment includes: the memory 710 and the processor 720 may further include an input/output interface 730, a network interface 740, a storage interface 750, and the like. These interfaces 730, 740, 750, as well as the memory 710 and the processor 720, may be connected, for example, by a bus 760. The input/output interface 730 provides a connection interface for input/output devices such as a display, a mouse, a keyboard, and a touch screen. The network interface 740 provides a connection interface for various networking devices. The storage interface 750 provides a connection interface for external storage devices such as an SD card and a usb disk.
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement any one of the foregoing capacity expansion methods.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.
Claims (18)
1. A capacity expansion method comprises the following steps:
acquiring current index data and historical index data;
predicting index data of the future preemption time by using the current index data and the historical index data;
determining the total capacity expansion according to the predicted index data of the preemption time;
and expanding the capacity according to the total capacity expansion amount.
2. The capacity expansion method according to claim 1, wherein the determining a total capacity expansion amount based on the predicted indicator data of the preemption time includes:
determining a resource increment corresponding to the preemption time according to the predicted index data of the preemption time;
determining a capacity expansion coefficient according to the resource increment;
predicting the capacity expansion probability of the preemption time by using the current index data, the historical index data and the historical capacity expansion event;
and determining the total capacity expansion amount according to the resource increment corresponding to the preemption time, the capacity expansion coefficient and the predicted capacity expansion probability of the preemption time.
3. The capacity expansion method of claim 2, wherein the index data comprises at least one of a user access amount, a resource usage amount of a service, or a cluster resource remaining amount.
4. The capacity expansion method of claim 3, wherein the determining a resource increment corresponding to the preemption time comprises:
determining a predicted user growth coefficient according to the historical user access amount and the user access amount of the preemption time;
determining a predicted service growth coefficient according to the historical service resource usage and the service resource usage of the seizing time;
and determining the resource increment corresponding to the preemption time according to the maximum value of the user growth coefficient and the service resource usage amount of the preemption time.
5. The capacity expansion method of claim 3, wherein the determining a capacity expansion coefficient according to the resource increment comprises:
and determining a capacity expansion coefficient according to the ratio of the resource increment to the predicted cluster resource surplus amount.
6. The capacity expansion method of claim 5, wherein determining a capacity expansion coefficient based on the ratio of the resource increment to the predicted remaining amount of cluster resources comprises:
determining the capacity expansion coefficient as 0 under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is smaller than a preset lower limit; or,
determining a capacity expansion coefficient as the ratio of the resource increment to the predicted cluster resource residual amount under the condition that the ratio is between the preset lower limit and the preset upper limit; or,
and under the condition that the ratio of the resource increment to the predicted cluster resource residual quantity is greater than a preset upper limit, determining the capacity expansion coefficient as 1.
7. The capacity expansion method of any one of claims 2-6, wherein predicting a capacity expansion probability of a future preemption time using the current index data, the historical index data, and a historical capacity expansion event comprises:
and processing the current index data, the historical index data and the historical capacity expansion event by using a deep learning network to obtain the capacity expansion probability of the future preemption time.
8. The capacity expansion method according to any one of claims 2 to 6, wherein the determining a total capacity expansion amount according to the resource increment corresponding to the preemption time, the capacity expansion coefficient, and the predicted capacity expansion probability of the preemption time includes:
and determining the product of the resource increment corresponding to the preemption time, the expansion coefficient and the predicted expansion probability of the preemption time as the total expansion capacity.
9. The capacity expansion method of claim 1, wherein the predicting, using the current metric data and the historical metric data, metric data for future preemption times comprises:
and processing the current index data and the historical index data by using a deep learning network to obtain the index data of the future preemption time.
10. The capacity expansion method according to claim 1, wherein the current index data and the historical index data are acquired when a time interval between the current time and the last acquisition of the index data satisfies a preset time interval.
11. The capacity expansion method of claim 10, wherein the preemption time and the time interval are user configured in a markup language YAML file.
12. The capacity expansion method according to claim 1,
and expanding the capacity of the container of the edge computing node according to the total expansion capacity.
13. A capacity expansion device comprising:
the acquisition module is configured to acquire current index data and historical index data;
a prediction module configured to predict future time-to-preemption target data using the current target data and the historical target data;
a determining module configured to determine a total capacity according to the predicted indicator data of the preemption time;
and the capacity expansion module is configured to expand the capacity according to the total capacity expansion amount.
14. A capacity expansion system, comprising:
the flash tank of claim 13;
a metrics server configured to query the current metric data; and
an index database configured to query historical index data.
15. The flash expansion system of claim 14, further comprising:
and the event server is configured to query the historical capacity expansion event.
16. The flash expansion system of claim 14, further comprising:
and the capacity expansion device expands the capacity of the edge computing node.
17. A capacity expansion device comprising:
a memory; and
a processor coupled to the memory, the processor configured to perform the capacity expansion method of any of claims 1-12 based on instructions stored in the memory.
18. A computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the capacity expansion method of any one of claims 1 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210680103.1A CN114866563A (en) | 2022-06-16 | 2022-06-16 | Capacity expansion method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210680103.1A CN114866563A (en) | 2022-06-16 | 2022-06-16 | Capacity expansion method, device, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114866563A true CN114866563A (en) | 2022-08-05 |
Family
ID=82624778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210680103.1A Pending CN114866563A (en) | 2022-06-16 | 2022-06-16 | Capacity expansion method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114866563A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115412567A (en) * | 2022-08-09 | 2022-11-29 | 浪潮云信息技术股份公司 | Cloud platform storage capacity planning system and method based on time series prediction |
| CN116980425A (en) * | 2023-06-21 | 2023-10-31 | 杭州瑞网广通信息技术有限公司 | Elastic telescoping method of streaming media cluster |
| CN117873632A (en) * | 2023-01-18 | 2024-04-12 | 苏州融拓数字智能科技有限公司 | Method and device for automatically expanding Pod in K8s according to traffic and computer storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766182A (en) * | 2018-12-18 | 2019-05-17 | 平安科技（深圳）有限公司 | The scalable appearance method, apparatus of system resource dynamic, computer equipment and storage medium |
| CN109918194A (en) * | 2019-01-16 | 2019-06-21 | 深圳壹账通智能科技有限公司 | Intelligent dilatation capacity reduction method, device, computer equipment and storage medium |
| CN110058939A (en) * | 2018-12-27 | 2019-07-26 | 阿里巴巴集团控股有限公司 | System Expansion method, device and equipment |
| CN114189482A (en) * | 2021-12-14 | 2022-03-15 | 郑州阿帕斯数云信息科技有限公司 | Control method, device and system for cluster resources |
| JP7079998B1 (en) * | 2021-12-16 | 2022-06-03 | 北京穿楊科技有限公司 | Cluster capacity expansion method and equipment |
-
2022
- 2022-06-16 CN CN202210680103.1A patent/CN114866563A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766182A (en) * | 2018-12-18 | 2019-05-17 | 平安科技（深圳）有限公司 | The scalable appearance method, apparatus of system resource dynamic, computer equipment and storage medium |
| CN110058939A (en) * | 2018-12-27 | 2019-07-26 | 阿里巴巴集团控股有限公司 | System Expansion method, device and equipment |
| CN109918194A (en) * | 2019-01-16 | 2019-06-21 | 深圳壹账通智能科技有限公司 | Intelligent dilatation capacity reduction method, device, computer equipment and storage medium |
| CN114189482A (en) * | 2021-12-14 | 2022-03-15 | 郑州阿帕斯数云信息科技有限公司 | Control method, device and system for cluster resources |
| JP7079998B1 (en) * | 2021-12-16 | 2022-06-03 | 北京穿楊科技有限公司 | Cluster capacity expansion method and equipment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115412567A (en) * | 2022-08-09 | 2022-11-29 | 浪潮云信息技术股份公司 | Cloud platform storage capacity planning system and method based on time series prediction |
| CN115412567B (en) * | 2022-08-09 | 2024-04-30 | 浪潮云信息技术股份公司 | Cloud platform storage capacity planning system and method based on time sequence prediction |
| CN117873632A (en) * | 2023-01-18 | 2024-04-12 | 苏州融拓数字智能科技有限公司 | Method and device for automatically expanding Pod in K8s according to traffic and computer storage medium |
| CN116980425A (en) * | 2023-06-21 | 2023-10-31 | 杭州瑞网广通信息技术有限公司 | Elastic telescoping method of streaming media cluster |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114866563A (en) | Capacity expansion method, device, system and storage medium | |
| US10572285B2 (en) | Method and apparatus for elastically scaling virtual machine cluster | |
| CN111026553B (en) | Resource scheduling method and server system for offline mixed part operation | |
| CN113835865B (en) | Task deployment method and device, electronic equipment and storage medium | |
| EP3129880A1 (en) | Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system | |
| CN106209967B (en) | A kind of video monitoring cloud resource prediction technique and system | |
| CN110764714B (en) | Data processing method, device and equipment and readable storage medium | |
| CN110990138A (en) | Resource scheduling method, device, server and storage medium | |
| CN108023834A (en) | A kind of cloud resource auto-allocation method and device | |
| KR20180072295A (en) | Dynamic job scheduling system and method for supporting real-time stream data processing in distributed in-memory environment | |
| CN114490078A (en) | Dynamic capacity reduction and expansion method, device and equipment for micro-service | |
| CN116643844B (en) | Intelligent management system and method for automatic expansion of power super-computing cloud resources | |
| CN111858067A (en) | Data processing method and device | |
| CN103488538B (en) | Application extension device and application extension method in cloud computing system | |
| CN112817721A (en) | Task scheduling method and device based on artificial intelligence, computer equipment and medium | |
| CN110688379B (en) | Data storage method, system, storage medium and electronic equipment | |
| CN108471362B (en) | Resource allocation prediction technique and device | |
| CN103678562A (en) | Capacity obtaining method and file data allocation method | |
| CN117519913B (en) | Method and system for elastically telescoping scheduling of container memory resources | |
| CN113900800B (en) | Distribution method of edge computing system | |
| CN108984344B (en) | Method, device and equipment for dynamically generating snapshot and storage medium | |
| CN113538026B (en) | Service amount calculation method and device | |
| CN116382848A (en) | Container management method, device, storage medium and electronic equipment | |
| CN115695433A (en) | Content distribution network, and adjusting method, device and storage medium thereof | |
| CN114443100A (en) | Update method, electronic device, and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |