CN110096339B - System load-based capacity expansion and contraction configuration recommendation system and method - Google Patents
System load-based capacity expansion and contraction configuration recommendation system and method Download PDFInfo
- Publication number
- CN110096339B CN110096339B CN201910389923.3A CN201910389923A CN110096339B CN 110096339 B CN110096339 B CN 110096339B CN 201910389923 A CN201910389923 A CN 201910389923A CN 110096339 B CN110096339 B CN 110096339B
- Authority
- CN
- China
- Prior art keywords
- load
- application
- configuration
- module
- load data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000008602 contraction Effects 0.000 title claims abstract description 27
- 238000011156 evaluation Methods 0.000 claims abstract description 12
- 238000013500 data storage Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 18
- 238000007726 management method Methods 0.000 claims description 15
- 238000013341 scale-up Methods 0.000 claims description 2
- 238000012795 verification Methods 0.000 claims description 2
- 238000004220 aggregation Methods 0.000 claims 1
- 230000002776 aggregation Effects 0.000 claims 1
- 238000012827 research and development Methods 0.000 abstract description 4
- 238000012423 maintenance Methods 0.000 abstract description 3
- 238000011160 research Methods 0.000 abstract 1
- 230000015654 memory Effects 0.000 description 22
- 238000005457 optimization Methods 0.000 description 16
- 238000005516 engineering process Methods 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007334 memory performance Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45579—I/O management, e.g. providing access to device drivers or storage
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45583—Memory management, e.g. access or allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45591—Monitoring or debugging support
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a system for recommending the configuration of expansion and contraction based on the system load, which comprises a load acquisition module, a load data receiving module and an application configuration recommending module, and is characterized in that: the load acquisition module is used for acquiring load data; the load data receiving module is used for summarizing the load data; and the application configuration recommending module is used for acquiring the recommended configuration of the expansion and contraction volume. The invention also discloses a method for automatically switching the flow based on the double-activity data center. The invention fully utilizes the data generated by each operation and maintenance system through the linkage of each system, dynamically expands and contracts the capacity, improves the resource utilization rate of various research and development environments and online environments, saves resources and reduces the cost consumption. Capacity evaluation prediction capability of research personnel is improved.
Description
Technical Field
The invention belongs to the technical field of computer science software information, and particularly relates to a system and a method for recommending expansion and contraction volume configuration based on system load.
Background
The completion of the online development of the system does not represent the end of the work, and the operation and maintenance of the online system are also the key points of the work. Since hypervisor virtualization technology still has some performance and resource usage efficiency problems, a new type of virtualization technology called Container technology (Container) has emerged to help solve these problems. Container technology virtualization technology has become a widely recognized way of sharing container technology server resources, and container technology can provide great flexibility for system administrators in the process of building container technology operating system instances on demand. When the virtual machine is used, the operation cost of capacity expansion and capacity reduction is extremely high. After the system is containerized, the research and development system is automatically expanded and contracted into functions, and the number of cores and the size of a memory of a CPU (central processing unit) used by an application can be specified. But exactly what configuration should be used depends on the evaluation and estimation of capacity by the skilled developer, and the human estimation will be biased. If the online capacity evaluation resources are insufficient, slow response of the application frequently occurs in the online running process, the application is forced to be closed by an operating system under the condition of serious shortage of the memory, and finally the service is unavailable, and the excessive resource evaluation causes waste of online machine resources.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a system for recommending the expansion and contraction capacity configuration based on the system load, which solves the problems of subjectivity and inaccuracy of application capacity evaluation.
In order to solve the technical problems, the invention adopts the following technical scheme:
the utility model provides a scale-up and scale-down configuration recommendation system based on system load realizes, it includes load collection module, load data receiving module and application configuration recommendation module, its characterized in that:
the load acquisition module is used for acquiring load data;
the load data receiving module is used for summarizing the load data;
and the application configuration recommending module is used for acquiring the recommended configuration of the expansion and contraction volume.
Wherein the system further comprises:
and the load data storage module is used for storing the load data summarized by the load data receiving module.
Wherein the system further comprises:
and the application type evaluation module is used for obtaining the application load ratio based on the application instance configuration and the application rating.
Wherein the system further comprises:
and the application expansion and contraction capacity module is used for calculating an expansion and contraction capacity difference ratio and a difference value based on the acquired expansion and contraction capacity recommended configuration.
Wherein the system further comprises:
a container management module to reconstruct an application instance based on the difference ratio and the difference value.
The load data receiving module is further used for sending the load data to a message middleware, periodically taking the load data out of the message middleware for summarizing and storing a summarizing result into the load data storage module.
The application configuration recommending module is specifically configured to obtain a data item by counting the summary result in the load data storage module, calculate a load ratio between the data item and the data item to obtain a recommended configuration item of the application, and obtain an optimal configuration value by adjusting the recommended configuration item.
Wherein the application scaling module is further configured to: and calling the application configuration recommending module through a triggering condition to obtain an optimal configuration value of the application and comparing the optimal configuration value with the current configuration value to obtain a difference ratio and a difference value.
Thus, a system for dynamically scaling the capacity according to the application example is provided, resources are saved, and cost consumption is reduced.
As optimization, the information of the application information module comprises a language used by an application, the current machine configuration condition and a current application machine instance list.
Therefore, all information related to the application can be provided, and the application load ratio can be obtained more comprehensively and accurately.
For optimization, the load data collected by the load collection module includes an entity machine load, a virtual machine load, a container load, a process resource occupation load in a container, and an application process resource occupation subdivision load.
Therefore, the load condition related to the application can be more comprehensively and accurately acquired.
As an optimization, the data items include the minimum, maximum, mean, standard deviation of the load.
Thus, the condition of the load value can be known more comprehensively.
And as optimization, the load data storage module is a time sequence database.
Therefore, the data monitored and collected in real time can be stored, and the requirements of the system can be further met.
As an optimization, in the application configuration recommendation module, the load proportion calculation corresponding to the data items is specifically as follows: and recommending a configuration item as a data item/load ratio corresponding to the data item.
Therefore, the value of the recommended configuration item can be obtained more accurately according to the load proportion corresponding to the data item and the value of the data item.
As an optimization, the triggering condition of the application scaling module is that the actual load of the application occupies more than 80% or less than 20% of the memory in one week or the average value of the CPU is more than 70% or less than 20%.
Therefore, the automatic triggering of the application expansion and contraction capacity module can be realized for the application with excessive load or insufficient load for a long time, and the automatic expansion and contraction capacity can be realized.
And as optimization, comparing the optimal configuration value with the current configuration value to obtain a difference ratio larger than 50% and a difference value larger than 100M, and reconstructing the application example by the container management module.
Thus, the compromise condition of constructing the application instance for the module easy to manage can meet the requirements of the storage with large memory and the storage with small memory at the same time.
As optimization, in the application configuration recommendation module, the method for adjusting the recommended configuration items includes taking the recommended configuration item with the largest value from the recommended configuration items, and rounding up to obtain the optimal configuration value.
Thus, the obtained optimal configuration value can be ensured to meet the maximum requirement of the application.
The invention also provides a method for recommending the expansion and contraction capacity configuration based on the system load, which comprises the following steps:
collecting load data;
summarizing the load data;
and acquiring recommended configuration of the expansion and contraction capacity.
Wherein the method further comprises:
and storing the load data summarized by the load data receiving module.
Wherein the method further comprises:
and obtaining the application load ratio based on the application example configuration and the application rating.
Wherein the method further comprises:
and calculating a scaling tolerance ratio and a difference value based on the obtained scaling capacity recommendation configuration.
Wherein the method further comprises:
reconstructing an application instance based on the difference ratio and the difference value.
And sending the load data to a message middleware, periodically taking the load data out of the message middleware for summarizing and storing a summarizing result.
And calculating the load proportion corresponding to the data items to obtain a recommended configuration item of the application, and adjusting the recommended configuration item to obtain an optimal configuration value.
The method comprises the steps of obtaining an optimal configuration value of an application through a trigger condition, and comparing the optimal configuration value with a current configuration value to obtain a difference ratio and a difference value.
The load data comprises a physical machine load, a virtual machine load, a container load, a process resource occupation load in the container and an application process resource occupation subdivision load.
Wherein the data items comprise minimum, maximum, mean, standard deviation of the load.
The load data storage module is a time sequence database.
The load proportion calculation of the data items and the data items is specifically as follows: and recommending a configuration item as a data item/load ratio corresponding to the data item.
The triggering condition is that the average occupied memory of the actual load is higher than 80% or lower than 20% or the average CPU value is higher than 70% or lower than 20% in one week.
And the container management module reconstructs the application example, wherein the difference ratio obtained by comparing the optimal configuration value with the current configuration value is more than 50% and the difference value is more than 100M.
The method for adjusting the recommended configuration items comprises the steps of taking the recommended configuration item with the largest value out of the recommended configuration items, and rounding up to obtain the optimal configuration value.
The invention has the beneficial effects that: according to the invention, through linkage of all systems, data generated by operation and maintenance of all systems are fully utilized, the capacity is dynamically expanded and contracted, the resource utilization rate of various research and development environments and online environments is improved, resources are saved, the cost consumption is reduced, and the capacity evaluation estimation capability of research and development personnel is improved.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
FIG. 1 is a system diagram of a system configuration recommendation system for implementing capacity expansion and contraction based on system load according to the present invention
Fig. 2 is a flowchart of a method for recommending the configuration of the scaling based on the system load according to the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
As shown in fig. 1, in a certain embodiment, the present invention provides a system for implementing capacity expansion and reduction configuration recommendation based on system load, which includes a load collection module, a load data receiving module, and an application configuration recommendation module, and is characterized in that:
the load acquisition module is used for acquiring load data;
the load data receiving module is used for summarizing the load data;
and the application configuration recommending module is used for acquiring the recommended configuration of the expansion and contraction volume.
Wherein the system further comprises:
and the load data storage module is used for storing the load data summarized by the load data receiving module.
Wherein the system further comprises:
and the application type evaluation module is used for obtaining the application load ratio based on the application instance configuration and the application rating.
Wherein the system further comprises:
and the application expansion and contraction capacity module is used for calculating an expansion and contraction capacity difference ratio and a difference value based on the acquired expansion and contraction capacity recommended configuration.
Wherein the system further comprises:
a container management module to reconstruct an application instance based on the difference ratio and the difference value.
The load data receiving module is further used for sending the load data to a message middleware, periodically taking the load data out of the message middleware for summarizing and storing a summarizing result into the load data storage module.
The application configuration recommending module is specifically configured to obtain a data item by counting the summary result in the load data storage module, calculate a load ratio between the data item and the data item to obtain a recommended configuration item of the application, and obtain an optimal configuration value by adjusting the recommended configuration item.
Wherein the application scaling module is further configured to: and calling the application configuration recommending module through a triggering condition to obtain an optimal configuration value of the application and comparing the optimal configuration value with the current configuration value to obtain a difference ratio and a difference value.
Thus, a system for dynamically scaling the capacity according to the application example is provided, resources are saved, and cost consumption is reduced.
As optimization, the information of the application information module comprises a language used by an application, the current machine configuration condition and a current application machine instance list.
Therefore, all information related to the application can be provided, and the application load ratio can be obtained more comprehensively and accurately.
For optimization, the load data collected by the load collection module includes an entity machine load, a virtual machine load, a container load, a process resource occupation load in a container, and an application process resource occupation subdivision load.
Therefore, the load condition related to the application can be more comprehensively and accurately acquired.
As an optimization, the data items include the minimum, maximum, mean, standard deviation of the load.
Thus, the condition of the load value can be known more comprehensively.
And as optimization, the load data storage module is a time sequence database.
Therefore, the data monitored and collected in real time can be stored, and the requirements of the system can be further met.
As an optimization, in the application configuration recommendation module, the load proportion calculation corresponding to the data items is specifically as follows: and recommending a configuration item as a data item/load ratio corresponding to the data item.
Therefore, the value of the recommended configuration item can be obtained more accurately according to the load proportion corresponding to the data item and the value of the data item.
As an optimization, the triggering condition of the application scaling module is that the actual load of the application occupies more than 80% or less than 20% of the memory in one week or the average value of the CPU is more than 70% or less than 20%.
Therefore, the automatic triggering of the application expansion and contraction capacity module can be realized for the application with excessive load or insufficient load for a long time, and the automatic expansion and contraction capacity can be realized.
And as optimization, comparing the optimal configuration value with the current configuration value to obtain a difference ratio larger than 50% and a difference value larger than 100M, and reconstructing the application example by the container management module.
Thus, the compromise condition of constructing the application instance for the module easy to manage can meet the requirements of the storage with large memory and the storage with small memory at the same time.
As optimization, in the application configuration recommendation module, the method for adjusting the recommended configuration items includes taking the recommended configuration item with the largest value from the recommended configuration items, and rounding up to obtain the optimal configuration value.
In another embodiment, the invention provides a system for implementing capacity expansion and reduction configuration recommendation based on system load, which comprises a load acquisition module, a load data receiving module, a load data storage module, an application type evaluation module, an application configuration recommendation module and an application capacity expansion module.
The load includes occupation of the CPU and the memory by the application.
The load acquisition module acquires load data through the performance monitoring module and uploads the acquired load data to the load data receiving module.
In this embodiment, the performance monitoring module includes a container monitoring Agent, a process resource occupation monitoring Agent in the container, and an application process resource occupation subdivision monitoring Agent. The load data comprises an entity machine load, a virtual machine load, a container load, a process resource occupation load in a container and an application process resource occupation subdivision load.
The load data receiving module receives the load data and sends the load data to the message middleware, the load data is taken out from the message middleware periodically for gathering, and the gathered result is stored in the load data storage module. In this embodiment, the message middleware may be kafka, rabbitmq or rocktmq. And the load data receiving module takes out the load data stored in the message middleware 30 seconds before every 30 seconds, and stores the load data into the load data storage module after summarizing. Therefore, load data are prevented from being stored in the load data storage module every second, and occupation of a system memory is reduced. The interval time may be other times.
And the load data storage module stores the summary result of the load data receiving module. In this embodiment, the load data storage module is a time sequence database.
And the application type evaluation module obtains the application load ratio through the information of the application information module, the application instance configuration and the application rating. The application will run on the new server according to the load ratio, and then look at the load of the application on the new server. In this embodiment, the information of the application information module includes a language, a size, a current machine configuration condition, and a current application machine instance list used by the application.
Specifically, the application type evaluation module evaluates the type of the application according to the application instance configuration, the load condition of the application and the language used by the application, and gives different application load ratios according to the application rating such as a core transaction application system.
The types of applications are classified as IO-intensive or CPU-intensive applications.
The CPU intensive type is also called compute intensive type, which means that the hard disk and memory performance of the system is much better than that of the CPU, at this time, most of the system operation conditions are that the CPU L loads 100%, the CPU needs to read/write I/O (hard disk/memory), I/O can be completed in a short time, the CPU also has many operations to process, and the CPU L loads very high.
In the multiprogramming system, most of the time is used for the calculation, logic determination and other CPU operations called CPU bound. For example, a program for calculating the circumference ratio to a decimal point of less than one thousand bits is a program belonging to the CPU bound, and most of the time is used for the calculation of the trigonometric function and the root opening during the execution process.
The programs of the CPU bound are generally quite high in CPU occupancy. This may be because the task itself does not need much access to the I/O device, or because the program is a multi-threaded implementation and thus masks the time waiting for I/O.
The IO-intensive type means that the CPU performance of the system is much better than that of a hard disk and a memory, at this time, the system is in operation, most conditions are that the CPU is in read/write operation of I/O (hard disk/memory), and at this time, the CPU L loading is not high.
When the program of the I/O bound generally reaches the performance limit, the CPU occupancy rate is still low. This may be because the task itself requires a large number of I/O operations, and pipeline does not perform well, not fully utilizing processor power.
Different languages are used by the applications, and different application load ratios also need to be used. Under general conditions, the memory occupation of java application is higher, the memory requirement of the application written by the PHP and Nodejs languages is not so high, and the CPU configuration proportion of the CPU of java and Nodejs and the CPU configuration proportion of PHP application can be different.
An application instance refers to a server that actually runs an application. The server may be a physical machine, or may be multiple virtual machines or containers running on a physical machine.
The application instance configuration includes information that the application runs several servers, each server has several cores and several G memories, application lists and the like.
The applied load condition can be obtained by counting the load value of one day of application, or by combining the application performance monitoring system to obtain the application interface call amount or user access amount in the same time period, the relationship between the call amount increase and decrease and the system load is calculated, the call amount trend and the data deviation condition in each day time period are obtained, and different flow time periods are divided. And calculating data items such as minimum values, maximum values, average values, standard deviations and the like of all load dimensions according to different time periods. In this embodiment, the application performance monitoring system is an APM system, and the call amount and the user access amount are collected by an Agent applied thereto.
The rating of an application may be manually rated by the importance of the application. In the present embodiment, the rating level is classified into A, B, C three levels from high to low. The load ratio of the grade A application is 30-40%, the load ratio of the grade B application is 40-60%, and the load ratio of the grade C application is 60-70%. The load matching and the rating can be gradually optimized through access machine learning.
The application configuration recommending module obtains the data items of the application by counting the summary result in the load data storage module, calculates the load proportion corresponding to the data items to obtain the recommended configuration items of the application, and obtains the optimal configuration value by adjusting the recommended configuration items. In this embodiment, the data items include a minimum value, a maximum value, an average value, and a standard deviation of the application load, and the recommended configuration item is a load ratio corresponding to the data item/data item. The load proportion corresponding to the data item can be obtained by two means: 1. continuously optimizing through manual experience; 2. and finding out the relation among the application, the access amount and the load ratio, and controlling by Ai to obtain the relation. The method for adjusting the recommended configuration items comprises the steps of taking the recommended configuration item with the largest value out of the recommended configuration items, and rounding up to obtain the optimal configuration value.
The method specifically comprises the following steps:
wherein,
in (1)
Represents rounding up on a; cpu on the right side of the formula 1 represents the load of the CPU occupied by the application, and the number of the CPUs is taken as a unit; mem to the right of equation 2 represents the load of the memory occupied by the application, in units of M. For example, if the load of the CPU occupied by the application is 150%, the optimal configuration value of the CPU is 2, and if the load of the memory occupied by the application is 750M, the optimal configuration value of the memory is 800M.
The application expansion and contraction capacity module calls the application configuration recommending module through the trigger condition to obtain the optimal configuration value of the application and compares the optimal configuration value with the current configuration value to obtain a difference ratio and a difference value, and the container management module reconstructs an application example according to the difference ratio and the difference value. In this embodiment, the triggering condition is that the average memory occupied by the actual load is higher than 80% or lower than 20% or the average CPU value is higher than 70% or lower than 20% in one week. The trigger condition may be defined according to the actual conditions of the enterprise, and may be other values. And the difference ratio obtained by comparing the optimal configuration value with the current configuration value is more than 50% and the difference value is more than 100M, and the container management module reconstructs the application example.
The difference ratio is abs (optimal configuration value-current configuration value)/optimal configuration value;
the difference is abs (best configuration value — current configuration value).
The container management module is a container management end, the container management end constructs a new application example again according to the optimal configuration value, and the old application example is gradually dropped after the new application example is initialized and the automatic verification service function is normal, so that the configuration switching is realized. And (4) periodically recalculating the configuration checking call which is not made recently, and if the difference ratio and the difference value between the optimal configuration value and the current configuration value do not meet the triggering condition, not performing the capacity expansion and contraction operation.
As shown in fig. 2, the present invention provides a method for recommending a scalable configuration based on system load, which includes:
collecting load data;
summarizing the load data;
and acquiring recommended configuration of the expansion and contraction capacity.
Wherein the method further comprises:
and storing the load data summarized by the load data receiving module.
Wherein the method further comprises:
and obtaining the application load ratio based on the application example configuration and the application rating.
Wherein the method further comprises:
and calculating a scaling tolerance ratio and a difference value based on the obtained scaling capacity recommendation configuration.
Wherein the method further comprises:
reconstructing an application instance based on the difference ratio and the difference value.
And sending the load data to a message middleware, periodically taking the load data out of the message middleware for summarizing and storing a summarizing result.
And calculating the load proportion corresponding to the data items to obtain a recommended configuration item of the application, and adjusting the recommended configuration item to obtain an optimal configuration value.
The method comprises the steps of obtaining an optimal configuration value of an application through a trigger condition, and comparing the optimal configuration value with a current configuration value to obtain a difference ratio and a difference value.
The load data comprises a physical machine load, a virtual machine load, a container load, a process resource occupation load in the container and an application process resource occupation subdivision load.
Wherein the data items comprise minimum, maximum, mean, standard deviation of the load.
The load data storage module is a time sequence database.
The load proportion calculation of the data items and the data items is specifically as follows: and recommending a configuration item as a data item/load ratio corresponding to the data item.
The triggering condition is that the average occupied memory of the actual load is higher than 80% or lower than 20% or the average CPU value is higher than 70% or lower than 20% in one week.
And the container management module reconstructs the application example, wherein the difference ratio obtained by comparing the optimal configuration value with the current configuration value is more than 50% and the difference value is more than 100M.
The method for adjusting the recommended configuration items comprises the steps of taking the recommended configuration item with the largest value out of the recommended configuration items, and rounding up to obtain the optimal configuration value.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (3)
1. The utility model provides a scale-up and scale-down configuration recommendation system based on system load realizes, it includes load collection module, load data receiving module and application configuration recommendation module, its characterized in that:
the load acquisition module is used for acquiring load data;
the load data receiving module is used for summarizing the load data;
the application configuration recommending module is used for acquiring recommended configuration of the expansion and contraction volume;
the system further comprises:
the load data storage module is used for storing the load data summarized by the load data receiving module;
the application type evaluation module is used for obtaining an application load ratio based on application instance configuration and application rating;
a container management module to reconstruct an application instance based on the difference ratio and the difference value;
the application configuration recommending module obtains a data item by counting the summary result in the load data storage module, calculates the load ratio of the data item to obtain a recommended configuration item of the application, and obtains an optimal configuration value by adjusting the recommended configuration item;
the load data comprises a physical machine load, a virtual machine load, a container load, a process resource occupation load in the container and an application process resource occupation subdivision load;
applying a scale and shrink module for: calling the application configuration recommending module through a triggering condition to obtain an optimal configuration value of the application and comparing the optimal configuration value with a current configuration value to obtain a difference ratio and a difference value;
the container management module is a container management end, the container management end constructs a new application example again according to the optimal configuration value, and the old application example is gradually dropped after the new application example is initialized and the automatic verification service function is normal, so that the configuration switching is realized; and recalculating the configuration checking call which is not made recently at regular intervals, and if the difference ratio and the difference value between the optimal configuration value and the current configuration value do not meet the triggering condition, not performing the capacity expansion and contraction operation.
2. The system according to claim 1, wherein the load data receiving module is further configured to send the load data to a message middleware, periodically take the load data from the message middleware for aggregation, and store the aggregated result in the load data storage module.
3. A method for implementing a system load-based scalable configuration recommendation implemented on the system of any one of claims 1-2, comprising: collecting load data;
summarizing the load data;
acquiring recommended configuration of the expansion and contraction capacity;
wherein the method further comprises:
storing the load data summarized by the load data receiving module;
wherein the method further comprises:
obtaining an application load ratio based on application instance configuration and application rating;
wherein the method further comprises:
calculating a scaling tolerance ratio and a difference value based on the obtained scaling capacity recommendation configuration;
wherein the method further comprises:
reconstructing an application instance based on the difference ratio and the difference value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910389923.3A CN110096339B (en) | 2019-05-10 | 2019-05-10 | System load-based capacity expansion and contraction configuration recommendation system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910389923.3A CN110096339B (en) | 2019-05-10 | 2019-05-10 | System load-based capacity expansion and contraction configuration recommendation system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110096339A CN110096339A (en) | 2019-08-06 |
| CN110096339B true CN110096339B (en) | 2020-08-04 |
Family
ID=67447705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910389923.3A Active CN110096339B (en) | 2019-05-10 | 2019-05-10 | System load-based capacity expansion and contraction configuration recommendation system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110096339B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382361B (en) * | 2020-03-12 | 2023-05-02 | 腾讯科技（深圳）有限公司 | Information pushing method, device, storage medium and computer equipment |
| CN111625195B (en) * | 2020-05-26 | 2023-11-07 | 北京百度网讯科技有限公司 | Method and device for server capacity expansion |
| CN112948127B (en) * | 2021-03-30 | 2023-11-10 | 北京滴普科技有限公司 | Cloud platform container average load monitoring method, terminal equipment and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1588318A (en) * | 2004-09-16 | 2005-03-02 | 上海交通大学 | Application server sample pool dynamic configuration method based on real number coding genetic calculation |
| CN102508693A (en) * | 2011-09-29 | 2012-06-20 | 华中科技大学 | Web server capacity expansion system based on virtual machine |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4755371B2 (en) * | 2001-09-25 | 2011-08-24 | 株式会社日立製作所 | Computer system |
| CN102681899B (en) * | 2011-03-14 | 2015-06-10 | 金剑 | Virtual computing resource dynamic management system of cloud computing service platform |
| CN103023969A (en) * | 2012-11-15 | 2013-04-03 | 北京搜狐新媒体信息技术有限公司 | Cloud platform scheduling method and system |
| CN104317638A (en) * | 2014-10-17 | 2015-01-28 | 华为技术有限公司 | Application stretching management method and device |
| CN106161625A (en) * | 2016-07-11 | 2016-11-23 | 浪潮(北京)电子信息产业有限公司 | A kind of method of the cloud computing resources utilization rate for improving virtual machine |
| CN106557353A (en) * | 2016-11-04 | 2017-04-05 | 天津轻工职业技术学院 | A kind of container carries the server performance index Evaluation Method of service application |
| CN109032805A (en) * | 2018-08-06 | 2018-12-18 | 深圳乐信软件技术有限公司 | A kind of scalable appearance method, apparatus of elasticity, server and storage medium |
| CN109144734A (en) * | 2018-09-12 | 2019-01-04 | 郑州云海信息技术有限公司 | A kind of container resource quota distribution method and device |
-
2019
- 2019-05-10 CN CN201910389923.3A patent/CN110096339B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1588318A (en) * | 2004-09-16 | 2005-03-02 | 上海交通大学 | Application server sample pool dynamic configuration method based on real number coding genetic calculation |
| CN102508693A (en) * | 2011-09-29 | 2012-06-20 | 华中科技大学 | Web server capacity expansion system based on virtual machine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110096339A (en) | 2019-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110096339B (en) | System load-based capacity expansion and contraction configuration recommendation system and method | |
| US9870269B1 (en) | Job allocation in a clustered environment | |
| WO2021159638A1 (en) | Method, apparatus and device for scheduling cluster queue resources, and storage medium | |
| US20150295970A1 (en) | Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system | |
| CN104915407A (en) | Resource scheduling method under Hadoop-based multi-job environment | |
| WO2020134364A1 (en) | Virtual machine migration method, cloud computing management platform, and storage medium | |
| CN115373835A (en) | Task resource adjusting method and device for Flink cluster and electronic equipment | |
| CN112579304A (en) | Resource scheduling method, device, equipment and medium based on distributed platform | |
| CN111966449B (en) | Virtual machine backup management method, system, terminal and storage medium | |
| CN102339233A (en) | Cloud computing centralized management platform | |
| CN107977167A (en) | Optimization method is read in a kind of degeneration of distributed memory system based on correcting and eleting codes | |
| CN112416568A (en) | Duration estimation method and duration estimation device for audio and video transcoding task | |
| CN110321364A (en) | Transaction data querying method, device and the terminal of credit card management system | |
| CN114490078A (en) | Dynamic capacity reduction and expansion method, device and equipment for micro-service | |
| CN110471769B (en) | Resource management method and device for virtual machine | |
| CN112000460A (en) | Service capacity expansion method based on improved Bayesian algorithm and related equipment | |
| CN117971488A (en) | Storage management method and related device for distributed database cluster | |
| CN112612604B (en) | Task scheduling method and device based on Actor model | |
| Lee et al. | Refining micro services placement over multiple kubernetes-orchestrated clusters employing resource monitoring | |
| CN116089477B (en) | Distributed training method and system | |
| US10594620B1 (en) | Bit vector analysis for resource placement in a distributed system | |
| CN116962532A (en) | Cluster task scheduling method and device, computer equipment and storage medium | |
| CN108595367B (en) | Server system based on computer cluster in local area network | |
| Skulysh et al. | Management of multiple stage queuing systems | |
| CN114598705B (en) | Message load balancing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |