CN108897673B - System capacity evaluation method and device - Google Patents
System capacity evaluation method and device Download PDFInfo
- Publication number
- CN108897673B CN108897673B CN201810731843.7A CN201810731843A CN108897673B CN 108897673 B CN108897673 B CN 108897673B CN 201810731843 A CN201810731843 A CN 201810731843A CN 108897673 B CN108897673 B CN 108897673B
- Authority
- CN
- China
- Prior art keywords
- capacity
- data
- determining
- operation data
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present disclosure provides a system capacity evaluation method and apparatus. The system capacity evaluation method comprises the following steps: monitoring system operation data and hardware operation data; determining system data capacity according to the system operation data; acquiring capacity correlation parameters of the system operation data and the hardware operation data; determining the physical capacity of the system according to the capacity correlation parameter; and determining the system capacity according to the system data capacity and the system physical capacity. The system capacity evaluation method can dynamically evaluate the system capacity in real time, and obtain a more accurate system capacity evaluation result under the conditions of not influencing the system operation and not increasing the cost.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a system capacity assessment method and apparatus capable of dynamically assessing system capacity in real time.
Background
With the coming of the internet + era, the SOA and micro-service architecture are deepened, the number of services is expanded continuously, the application environment is complicated, the service dependency relationship is changed continuously, and the real-time understanding of the system capacity condition and the evaluation of the system capacity become important targets.
In the related art, the system capacity is mainly evaluated by a line-down pressure measurement method and a line-up pressure measurement method. The online push-down test method is to copy online traffic directly to a test server through a tool, obtain the QPS (Query Per Second, Query rate Per Second) with the highest application when the test server is in a bottleneck, and calculate the system capacity on the line through a conversion coefficient below the online line. The online pressure measurement method is mainly characterized in that different weights are assigned to different servers during load polling, and the weight of one server is gradually increased, so that the flow of the server is far larger than that of other servers until the server has a performance bottleneck. This bottleneck may be a physical bottleneck such as CPU, LOAD, memory, bandwidth, etc., or a software bottleneck such as RT, failure rate, QPS fluctuation, etc. When the performance of the single machine is in a performance bottleneck, the application QPS at the moment is recorded as the single machine capacity, and the system capacity of the cluster is obtained according to the number of the cluster servers.
Both online pressure measurement and offline pressure measurement are time-consuming and labor-consuming, and reflect the system capacity during pressure measurement. Today, the internet is rapidly developing, the program version iteration speed is remarkable, and the capacity evaluation by performing one pressure measurement on each version iteration and environment change is unrealistic and not operable.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure is directed to a system capacity evaluation method and a system capacity evaluation apparatus for overcoming, at least to some extent, the problems of high voltage evaluation cost and low real-time performance due to the limitations and disadvantages of the related art.
According to a first aspect of the embodiments of the present disclosure, there is provided a system capacity evaluation method, including: monitoring system operation data and hardware operation data; determining system data capacity according to the system operation data; acquiring capacity correlation parameters of the system operation data and the hardware operation data; determining the physical capacity of the system according to the capacity correlation parameter; and determining the system capacity according to the system data capacity and the system physical capacity.
In an exemplary embodiment of the disclosure, the determining a system data capacity from the system operational data comprises:
determining system operation parameters according to the system operation data;
determining a plurality of data processing bottleneck values corresponding to one operation unit according to the system operation parameters, and taking the minimum value of the data processing bottleneck values as the unit capacity of the operation unit;
determining the occupation proportion of system resources according to system operation data and unit capacity of a plurality of operation units in the system;
and determining the system data capacity according to the proportion of the system operation data to the system resource occupation.
In an exemplary embodiment of the disclosure, the determining a plurality of data processing bottleneck values corresponding to one operation unit according to the system operation parameters includes:
acquiring task execution parameters corresponding to a plurality of task types according to the system operation parameters;
determining a maximum number of executions per second corresponding to each of the task categories according to the task execution parameters.
In an exemplary embodiment of the disclosure, the determining a system resource occupation ratio according to system operation data and unit capacity of a plurality of operation units in a system includes:
determining a first resource occupation proportion of each operation unit according to the current operation values of the operation units and the corresponding unit capacity;
and taking the sum of the first resource occupation proportions of the plurality of operation units as the system resource occupation proportion.
In an exemplary embodiment of the disclosure, the obtaining the capacity correlation parameter of the system operation data and the hardware operation data includes:
fitting the system operation data and the hardware operation data according to a plurality of fitting methods to obtain a plurality of fitting results;
and determining the incidence relation between the system operation data and the hardware operation data according to the fitting method corresponding to the maximum value of the fitting parameters in the fitting results, and setting the parameters corresponding to the fitting method as capacity correlation parameters.
In an exemplary embodiment of the present disclosure, the determining the system physical capacity according to the capacity correlation parameter includes:
determining a plurality of physical bottleneck values according to the system operation data and the capacity correlation parameter; setting a minimum value of the plurality of physical bottleneck values to the system physical capacity.
In an exemplary embodiment of the present disclosure, the determining a system capacity according to the system data capacity and the system physical capacity includes:
and taking the small value of the system data capacity and the system physical capacity as the system capacity.
According to a second aspect of the embodiments of the present disclosure, there is provided a system capacity evaluation apparatus, including:
the data monitoring module is used for monitoring system operation data and hardware operation data;
a software capacity evaluation module configured to determine a system data capacity based on the system operational data;
a correlation evaluation module configured to obtain a capacity correlation parameter of the system operation data and the hardware operation data;
a hardware capacity evaluation module configured to determine a system physical capacity based on the capacity correlation parameter;
and the comprehensive evaluation module is set to determine the system capacity according to the system data capacity and the system physical capacity.
According to a third aspect of the present disclosure, there is provided a system capacity evaluation apparatus including: a memory; and a processor coupled to the memory, the processor configured to perform the method of any of the above based on instructions stored in the memory.
According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a program which, when executed by a processor, implements a system capacity assessment method as recited in any of the above.
According to the embodiment of the disclosure, the system performance index is calculated according to the latest data at regular time by monitoring the system operation data and the hardware operation data, so that the data processing capacity and the physical capacity of the system are obtained, and the real-time capacity of the system can be calculated timely and effectively. Because manual participation is not needed, the pressure measurement cost is greatly reduced, the pressure measurement efficiency is improved, the real-time performance of data is ensured, and the constantly changing program running environment can be effectively dealt with.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.
Fig. 1 is a flow chart of a system capacity evaluation method in an exemplary embodiment of the present disclosure.
Fig. 2 is a sub-flowchart of a system capacity assessment method in an exemplary embodiment of the present disclosure.
Fig. 3 is a sub-flowchart of a system capacity assessment method in an exemplary embodiment of the present disclosure.
Fig. 4 is a sub-flowchart of a system capacity assessment method in an exemplary embodiment of the present disclosure.
Fig. 5 is a sub-flowchart of a system capacity assessment method in an exemplary embodiment of the present disclosure.
Fig. 6 is a sub-flowchart of a system capacity assessment method in an exemplary embodiment of the present disclosure.
Fig. 7 is a block diagram of a system capacity estimation apparatus in an exemplary embodiment of the present disclosure.
FIG. 8 is a block diagram of an electronic device in an exemplary embodiment of the disclosure.
FIG. 9 is a schematic illustration of a computer-readable storage medium in an exemplary embodiment of the disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Further, the drawings are merely schematic illustrations of the present disclosure, in which the same reference numerals denote the same or similar parts, and thus, a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The following detailed description of exemplary embodiments of the disclosure refers to the accompanying drawings.
Fig. 1 schematically shows a flow chart of a system capacity evaluation method in an exemplary embodiment of the present disclosure. Referring to fig. 1, a system capacity assessment method 100 may include:
step S1, monitoring system operation data and hardware operation data;
step S2, determining the system data capacity according to the system operation data;
step S3, acquiring capacity correlation parameters of the system operation data and the hardware operation data;
step S4, determining the system physical capacity according to the capacity correlation parameter;
step S5, determining system capacity according to the system data capacity and the system physical capacity.
According to the embodiment of the disclosure, the system performance index is calculated according to the latest data at regular time by monitoring the system operation data and the hardware operation data, so that the data processing capacity and the physical capacity of the system are obtained, and the real-time capacity of the system can be calculated timely and effectively. Because manual participation is not needed, the pressure measurement cost is greatly reduced, the pressure measurement efficiency is improved, the real-time performance of data is ensured, and the constantly changing program running environment can be effectively dealt with.
Next, each step of the system capacity estimation method 100 will be described in detail.
In step S1, system operational data and hardware operational data are monitored.
In the embodiments of the present disclosure, the system operation data may include, for example, database access data, database connection data, thread parameters, business logic consumption data, and the like; the hardware operational data may include CPU usage, network bandwidth occupancy, and the like.
The system operation data and the hardware operation data can be obtained at regular time according to the preset period, so that the timeliness of the data is ensured, and the latest system capacity evaluation result can be obtained in time under the continuously changing program operation environment such as continuous iteration of the application program version, continuous upgrading of other services depending on the system and the like.
At step S2, a system data capacity is determined based on the system operational data.
Fig. 2 is a sub-flowchart of step S2.
Referring to fig. 2, in an exemplary embodiment of the present disclosure, step S2 may include:
step S21, determining system operation parameters according to the system operation data;
step S22, determining a plurality of data processing bottleneck values corresponding to an operation unit according to the system operation parameters, and taking the minimum value of the plurality of data processing bottleneck values as the unit capacity of the operation unit;
step S23, determining the proportion of system resource occupation according to the system operation data and unit capacity of a plurality of operation units in the system;
step S24, determining the system data capacity according to the proportion of the system operation data and the system resource occupation.
The system operation parameters are, for example, the number of database accesses per second, the database access time, the service logic execution time, and the like. The method of determining the system operating parameters from the system operating data may be, for example, averaging. For example, if the number of database accesses within 10 seconds is determined to be 60 times and each access time is determined to be 6ms, 10ms, … …, 14ms, etc., according to the system operation data, the number of database accesses per second may be set to 6, and the database access time may be 10ms as the average value of each access time. The system operation parameters can be various, and those skilled in the art can set the type and calculation method of the calculated system operation parameters according to the actual situation.
Fig. 3 is a sub-flowchart of step S22.
Referring to fig. 3, in a disclosed embodiment, step S22 may include:
step S221, acquiring task execution parameters corresponding to a plurality of task types according to the system operation parameters;
step S222, determining the maximum execution times per second corresponding to each task type according to the task execution parameters.
For the process of evaluating the system data capacity, in the embodiment of the present disclosure, an application executed by the system and a plurality of running units included in the application may be determined first, where the running units are, for example, methods (methods) corresponding to the application. Next, determining the time consumption details corresponding to each method according to the system operation parameters calculated in the previous step.
For example, if a method is in a certain sampling time, the average QPS is 200, the average elapsed time is 100ms, the corresponding database access times per second is 6, each time takes 10ms, that is, the total database consumption time is 60ms, and the service logic takes 40 ms. The time consumption of the database and the service logic is two different task types, and 6 times, 10ms, 40ms and the like are task execution parameters corresponding to the task types.
If the maximum number of connections in the database connection pool is 30, and the maximum number of thread pools for executing the method is 50 (for simplicity, the switching cost of threads is not considered for the moment), then the database has a highest single-machine QPS (data processing bottleneck value) of 30 × 1000/60 — 500 times, and the highest single-machine QPS (data processing bottleneck value) of the service logic has a highest single-machine QPS (data processing bottleneck value) of 50 × 1000/40 — 1250 times, it is obvious that the bottleneck point of the method is on the database, that is, the highest single-machine QPS of the method is 500 times, that is, the unit capacity (maximum number of times the method is executed per second) corresponding to the method is 500 times.
If the method is optimized through software upgrading and other modes, the time consumption of each access of the database is reduced to 5ms, the average access times are changed into 4 times, namely the total time consumption of the database is changed into 20ms, the service logic time consumption is still 40ms, the highest single-machine QPS of the database is 30 × 1000/20 to 1500 times, obviously, the bottleneck point is in the service logic, namely the highest single-machine QPS of the method is 1250 times.
Fig. 4 is a sub-flowchart of step S23.
Referring to fig. 4, in a disclosed embodiment, step S23 may include:
step S231, determining a first resource occupation ratio of each of the operation units according to the current operation values of the operation units and the corresponding unit capacities;
step S232, taking the sum of the first resource occupation ratios of the plurality of operation units as the system resource occupation ratio.
After the unit capacity corresponding to each method is determined, the corresponding first resource occupation ratio can be determined according to the current operation value corresponding to the method in the system operation data. E.g. cell capacity C as per method ii500 times, and the current operation value TPS (Transaction Per Second) of the method is 100 times, the first resource occupation ratio P corresponding to the method ii＝Ti/Ci100/500 ═ 100 ═ 20%, where TiTPS value for method i.
Since one application program corresponds to a plurality of methods, the sum of the first resource occupation ratios of each method can be used as the system resource occupation ratio, where P ═ Σ Pi＝Σ[Ti/Ci]Wherein P is the occupation ratio of system resources. For example, P for three methods included by an applicationi20%, 15%, 25%, the system resource occupation ratio at this time is 60%.
If the single machine TPS of the current moment of the application program is obtained according to the system operation data as TAIf CPU and network bandwidth are not considered for the moment, the system data capacity corresponding to the application program is the system data capacityCapacity of a single machine CA＝TA/P。
In step S3, a capacity correlation parameter between the system operation data and the hardware operation data is obtained.
Fig. 5 is a sub-flowchart of step S3.
Referring to fig. 5, in a disclosed embodiment, step S3 may include:
step S31, fitting the system operation data and the hardware operation data according to a plurality of fitting methods to obtain a plurality of fitting results;
step S32, determining the incidence relation between the system operation data and the hardware operation data according to the fitting method corresponding to the maximum value of the fitting parameters in the fitting results, and setting the parameters corresponding to the fitting method as capacity correlation parameters.
Besides the system data capacity corresponding to the software, the system physical capacity corresponding to the hardware is also an important index for restricting the system capacity. In order to determine the correlation between the system physical capacity corresponding to the hardware and the system capacity, in the embodiment of the present disclosure, a fitting method is used to analyze the system operation data and the hardware operation data, so as to obtain an association formula, thereby further determining the system physical capacity.
In order to determine which fitting method can obtain an accurate correlation formula, a plurality of fitting methods can be used for fitting a data matrix comprising system operation data and hardware operation data, the fitting method is selected according to parameters in a fitting result, and each data column in the data matrix corresponds to one data type.
For example, a sample correlation coefficient matrix may be generated using the function corrcoef of MATLAB for a data matrix including CPU usage data and application TPS corresponding to its time instant, and the correlation coefficient may range from-1 to 1:
when the correlation coefficient is close to 1, the data columns have positive linear relation, namely positive correlation;
when the correlation coefficient is close to-1, the negative linear relation, namely the inverse correlation, exists between the data columns;
when the correlation coefficient is close to or equal to 0, it indicates an almost wireless relationship between the data columns.
After loading the data samples (e.g., count. dat), a correlation coefficient matrix is calculated by corrcoef (count), assuming that the resulting correlation coefficient matrix is as follows:
1.0000 0.9588
0.9588 1.0000
in the matrix, all correlation coefficients are close to 1, that is, each pair of data columns of the sample data has strong forward correlation, and a linear regression analysis prediction method can be adopted to fit the sample set. The fitting equation obtained by the unary linear regression analysis may be
Wherein x istThe value representing the argument i.e. the CPU utilization,
the values representing the dependent variables, i.e. TPS, a, b represent the parameters of a one-dimensional linear regression equation.
In some embodiments, if the correlation coefficient in the correlation coefficient matrix calculated by corrcoef (count) is less than a threshold (e.g., 0.8), it indicates that there is no strong linear relationship between the data columns. A non-linear curve fit may be used at this time. The curve fitting may be performed using a variety of approximation methods including, but not limited to, exponential approximation, fourier approximation, gaussian approximation, interpolation approximation, multi-form approximation, power approximation, rational number approximation, smooth approximation, sinusoidal approximation, and the like, and the fitting parameters may be compared.
For example, using Fourier approximation
The resulting fitting parameters may be:
SSE:0.02709
R-square:0.9978
Adjusted R-square:0.9913
RMSE:0.1164
where R-square is the deterministic coefficient of the curve equation, which characterizes the closeness of a fit, the closer to 1, the higher the closeness of the model to the data fit.
If the R-square in the above results is closest to 1 in multiple fitting modes, it means that Fourier approximation is the best fitting method to fit the sample set. At this time, a capacity correlation parameter, i.e., a correlation between the system physical capacity and the hardware operation data, may be determined according to the fitting result.
In step S4, a system physical capacity is determined based on the capacity correlation parameter.
Fig. 6 is a sub-flowchart of step S4.
Referring to fig. 6, in a disclosed embodiment, step S4 may include:
step S41, determining a plurality of physical bottleneck values according to the system operation data and the capacity correlation parameter;
step S42, setting the minimum value of the plurality of physical bottleneck values as the system physical capacity.
A physical bottleneck value may be calculated for each type of hardware operational data based on the system operational data. For example, a corresponding TPS value, i.e., QPS, when the CPU usage is 100% may be calculated from the CPU usage and the capacity correlation parameter. Similarly, physical bottleneck values of other physical resources such as network bandwidth can also be obtained, and the minimum value in the physical bottleneck values is set as the physical capacity of the system.
In step S5, a system capacity is determined according to the system data capacity and the system physical capacity.
In the embodiment of the present disclosure, a small value of the system data capacity and the system physical capacity may be used as the system capacity. For example, the system data capacity is 500 times, the system physical capacity is 400 times, and then the maximum QPS of the system is limited to the system physical capacity being only 400 times, and the current physical environment is unable to bear the larger QPS.
By determining the system capacity according to the system data capacity and the system physical capacity, it can be analyzed what factors restrict the system capacity, so as to make a targeted improvement plan.
According to the capacity planning method and device, the capacity of the system is dynamically calculated by monitoring the system operation data and the hardware operation data in real time, the capacity evaluation can be carried out in place of a pressure measurement mode, and real-time capacity planning is achieved. The problems of high real-time performance, high labor cost, high physical cost, high time cost and the like of the conventional pressure measurement mode are solved.
Corresponding to the above method embodiments, the present disclosure also provides a system capacity evaluation apparatus, which may be used to execute the above method embodiments.
Fig. 7 schematically shows a block diagram of a system capacity evaluation apparatus in an exemplary embodiment of the present disclosure.
Referring to fig. 7, the system capacity evaluating apparatus 700 may include:
a data monitoring module 71 configured to monitor system operation data and hardware operation data;
a software capacity assessment module 72 arranged to determine system data capacity from the system operational data;
a correlation evaluation module 73 configured to obtain a capacity correlation parameter of the system operation data and the hardware operation data;
a hardware capacity assessment module 74 configured to determine a system physical capacity based on the capacity correlation parameter;
a comprehensive evaluation module 75 configured to determine a system capacity based on the system data capacity and the system physical capacity.
In an exemplary embodiment of the present disclosure, the software capacity evaluation module 72 includes:
a parameter determination unit 721 arranged to determine system operation parameters from said system operation data;
a unit capacity determining unit 722 configured to determine a plurality of data processing bottleneck values corresponding to one operating unit according to the system operating parameters, and take a minimum value of the plurality of data processing bottleneck values as a unit capacity of the operating unit;
a resource occupancy rate determining unit 723, configured to determine a system resource occupancy ratio according to system operation data of a plurality of operation units in the system and the unit capacity;
a data capacity determining unit 724 configured to determine the system data capacity according to the ratio of the system operating data to the system resource occupation ratio.
In an exemplary embodiment of the present disclosure, the unit capacity determination unit 722 is configured to obtain task execution parameters corresponding to a plurality of task categories according to the system operation parameters; determining a maximum number of executions per second corresponding to each of the task categories according to the task execution parameters.
In an exemplary embodiment of the present disclosure, the resource occupancy determination unit 723 is configured to: determining a first resource occupation proportion of each operation unit according to the current operation values of the operation units and the corresponding unit capacity; and taking the sum of the first resource occupation proportions of the plurality of operation units as the system resource occupation proportion.
In an exemplary embodiment of the present disclosure, the relevance evaluation module 73 is configured to: fitting the system operation data and the hardware operation data according to a plurality of fitting methods to obtain a plurality of fitting results; and determining the incidence relation between the system operation data and the hardware operation data according to the fitting method corresponding to the maximum value of the fitting parameters in the fitting results, and setting the parameters corresponding to the fitting method as capacity correlation parameters.
In an exemplary embodiment of the present disclosure, the hardware capacity assessment module 74 is configured to determine a plurality of physical bottleneck values based on the system operational data and the capacity correlation parameter; setting a minimum value of the plurality of physical bottleneck values to the system physical capacity.
In an exemplary embodiment of the present disclosure, the comprehensive evaluation module 75 is configured to take a small value of the system data capacity and the system physical capacity as the system capacity.
Since the functions of the apparatus 700 have been described in detail in the corresponding method embodiments, the disclosure is not repeated herein.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 800 according to this embodiment of the invention is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present invention.
As shown in fig. 8, electronic device 800 is in the form of a general purpose computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one memory unit 820, and a bus 830 that couples the various system components including the memory unit 820 and the processing unit 810.
Wherein the storage unit stores program code that is executable by the processing unit 810 to cause the processing unit 810 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification. For example, the processing unit 810 may perform step S1 as shown in fig. 1: monitoring system operation data and hardware operation data; step S2: determining system data capacity according to the system operation data; step S3: acquiring capacity correlation parameters of the system operation data and the hardware operation data; step S4: and determining the physical capacity of the system according to the capacity correlation parameter.
The storage unit 820 may include readable media in the form of volatile memory units such as a random access memory unit (RAM)8201 and/or a cache memory unit 8202, and may further include a read only memory unit (ROM) 8203.
The storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 800 may also communicate with one or more external devices 600 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 800, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 800 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 850. Also, the electronic device 800 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 860. As shown, the network adapter 860 communicates with the other modules of the electronic device 800 via the bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.
Referring to fig. 9, a program product 900 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
Claims (8)
1. A method for system capacity assessment, comprising:
monitoring system operation data and hardware operation data;
determining system data capacity according to the system operation data;
acquiring capacity correlation parameters of the system operation data and the hardware operation data;
determining the physical capacity of the system according to the capacity correlation parameter;
determining system capacity according to the system data capacity and the system physical capacity;
wherein said determining system data capacity from said system operational data comprises: determining system operation parameters according to the system operation data; determining a plurality of data processing bottleneck values corresponding to one operation unit according to the system operation parameters, and taking the minimum value of the data processing bottleneck values as the unit capacity of the operation unit; determining the occupation proportion of system resources according to system operation data and unit capacity of a plurality of operation units in the system; determining the system data capacity according to the proportion of the system operation data to the system resource occupation;
the determining the proportion of the system resource occupation according to the system operation data and the unit capacity of a plurality of operation units in the system comprises the following steps: determining a first resource occupation proportion of each operation unit according to the current operation values of the operation units and the corresponding unit capacity; and taking the sum of the first resource occupation proportions of the plurality of operation units as the system resource occupation proportion.
2. The method for system capacity assessment according to claim 1, wherein said determining a plurality of data processing bottleneck values corresponding to an operating unit based on said system operating parameters comprises:
acquiring task execution parameters corresponding to a plurality of task types according to the system operation parameters;
determining a maximum number of executions per second corresponding to each of the task categories according to the task execution parameters.
3. The system capacity estimation method of claim 1, wherein the obtaining the capacity correlation parameter of the system operation data and the hardware operation data comprises:
fitting the system operation data and the hardware operation data according to a plurality of fitting methods to obtain a plurality of fitting results;
and determining the incidence relation between the system operation data and the hardware operation data according to the fitting method corresponding to the maximum value of the fitting parameters in the fitting results, and setting the parameters corresponding to the fitting method as capacity correlation parameters.
4. The system capacity estimation method of claim 3, wherein the determining system physical capacity based on the capacity correlation parameter comprises:
determining a plurality of physical bottleneck values according to the system operation data and the capacity correlation parameter;
setting a minimum value of the plurality of physical bottleneck values to the system physical capacity.
5. The system capacity estimation method of claim 1, wherein the determining system capacity based on the system data capacity and the system physical capacity comprises:
and taking the small value of the system data capacity and the system physical capacity as the system capacity.
6. A system capacity estimation apparatus, comprising:
the data monitoring module is used for monitoring system operation data and hardware operation data;
a software capacity evaluation module configured to determine a system data capacity based on the system operational data;
a correlation evaluation module configured to obtain a capacity correlation parameter of the system operation data and the hardware operation data;
a hardware capacity evaluation module configured to determine a system physical capacity based on the capacity correlation parameter;
a comprehensive evaluation module configured to determine a system capacity based on the system data capacity and the system physical capacity;
wherein the software capacity assessment module is configured to: determining system operation parameters according to the system operation data; determining a plurality of data processing bottleneck values corresponding to one operation unit according to the system operation parameters, and taking the minimum value of the data processing bottleneck values as the unit capacity of the operation unit; determining the occupation proportion of system resources according to system operation data and unit capacity of a plurality of operation units in the system; determining the system data capacity according to the proportion of the system operation data to the system resource occupation;
the determining the proportion of the system resource occupation according to the system operation data and the unit capacity of a plurality of operation units in the system comprises the following steps: determining a first resource occupation proportion of each operation unit according to the current operation values of the operation units and the corresponding unit capacity; and taking the sum of the first resource occupation proportions of the plurality of operation units as the system resource occupation proportion.
7. An electronic device, comprising:
a memory; and
a processor coupled to the memory, the processor configured to perform the system capacity assessment method of any of claims 1-5 based on instructions stored in the memory.
8. A computer-readable storage medium on which a program is stored, which program, when executed by a processor, implements the system capacity estimation method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810731843.7A CN108897673B (en) | 2018-07-05 | 2018-07-05 | System capacity evaluation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810731843.7A CN108897673B (en) | 2018-07-05 | 2018-07-05 | System capacity evaluation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108897673A CN108897673A (en) | 2018-11-27 |
| CN108897673B true CN108897673B (en) | 2022-04-12 |
Family
ID=64347777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810731843.7A Active CN108897673B (en) | 2018-07-05 | 2018-07-05 | System capacity evaluation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108897673B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581070B (en) * | 2020-05-07 | 2023-08-18 | 拉扎斯网络科技（上海）有限公司 | Capacity determination method, device, electronic equipment and computer readable storage medium |
| CN112365003B (en) * | 2020-11-16 | 2023-04-11 | 浙江百应科技有限公司 | Method for adjusting NLP model capacity based on big data |
| CN115292146B (en) * | 2022-05-30 | 2023-04-11 | 北京结慧科技有限公司 | System capacity estimation method, system, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866408A (en) * | 2014-02-20 | 2015-08-26 | 阿里巴巴集团控股有限公司 | Capacity prediction method and device for application system |
| CN105446905A (en) * | 2014-09-01 | 2016-03-30 | 炬力集成电路设计有限公司 | Method, equipment and system for evaluating transmission performance |
| CN106886485A (en) * | 2017-02-28 | 2017-06-23 | 深圳市华傲数据技术有限公司 | Power system capacity analyzing and predicting method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120290264A1 (en) * | 2011-05-12 | 2012-11-15 | Fluke Corporation | Method and apparatus for dynamically adjusting data acquisition rate in an apm system |
-
2018
- 2018-07-05 CN CN201810731843.7A patent/CN108897673B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866408A (en) * | 2014-02-20 | 2015-08-26 | 阿里巴巴集团控股有限公司 | Capacity prediction method and device for application system |
| CN105446905A (en) * | 2014-09-01 | 2016-03-30 | 炬力集成电路设计有限公司 | Method, equipment and system for evaluating transmission performance |
| CN106886485A (en) * | 2017-02-28 | 2017-06-23 | 深圳市华傲数据技术有限公司 | Power system capacity analyzing and predicting method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108897673A (en) | 2018-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9696786B2 (en) | System and method for optimizing energy consumption by processors | |
| CN110852421B (en) | Model generation method and device | |
| CN108509325B (en) | Method and device for dynamically determining system timeout time | |
| JP6193393B2 (en) | Power optimization for distributed computing systems | |
| US9471375B2 (en) | Resource bottleneck identification for multi-stage workflows processing | |
| CN108897673B (en) | System capacity evaluation method and device | |
| CN109272348B (en) | Method and device for determining number of active users, storage medium and electronic equipment | |
| EP2040170A1 (en) | System configuration parameter set optimizing method, program, and device | |
| CN111881023B (en) | Software aging prediction method and device based on multi-model comparison | |
| US11204851B1 (en) | Real-time data quality analysis | |
| Liu et al. | CSSAP: Software aging prediction for cloud services based on ARIMA-LSTM hybrid model | |
| CN113837596B (en) | Fault determination method and device, electronic equipment and storage medium | |
| KR102038703B1 (en) | Method for estimation on online multivariate time series using ensemble dynamic transfer models and system thereof | |
| US8832839B2 (en) | Assessing system performance impact of security attacks | |
| Chen et al. | Optimal variability sensitive condition-based maintenance with a Cox PH model | |
| CN112508695A (en) | Financial risk prediction method and device based on market risk and electronic equipment | |
| CN110727437A (en) | Code optimization item acquisition method and device, storage medium and electronic equipment | |
| CN112508723A (en) | Financial risk prediction method and device based on automatic preferential modeling and electronic equipment | |
| CN108804601A (en) | Power grid operation monitors the active analysis method of big data and device | |
| CN111897706A (en) | Server performance prediction method, device, computer system and medium | |
| CN114818913A (en) | Decision generation method and device | |
| CN117370065B (en) | Abnormal task determining method, electronic equipment and storage medium | |
| US20220035721A1 (en) | Efficient real-time data quality analysis | |
| CN112488865A (en) | Financial risk prediction method and device based on financial time nodes and electronic equipment | |
| CN110413482B (en) | Detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Beijing Economic and Technological Development Zone, 100176 Applicant before: BEIJING JINGDONG FINANCIAL TECHNOLOGY HOLDING Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |