CN115270603A - Server power consumption modeling method based on multivariate fine-grained feature regression analysis - Google Patents
Server power consumption modeling method based on multivariate fine-grained feature regression analysis Download PDFInfo
- Publication number
- CN115270603A CN115270603A CN202210751498.XA CN202210751498A CN115270603A CN 115270603 A CN115270603 A CN 115270603A CN 202210751498 A CN202210751498 A CN 202210751498A CN 115270603 A CN115270603 A CN 115270603A
- Authority
- CN
- China
- Prior art keywords
- power consumption
- server
- characteristic
- modeling
- regression analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/02—Reliability analysis or reliability optimisation; Failure analysis, e.g. worst case scenario performance, failure mode and effects analysis [FMEA]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/06—Power analysis or power optimisation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a server power consumption modeling method based on multivariate fine-grained characteristic regression analysis, which considers the fine-grained characteristics of inherent hardware and application programs of a server and constructs an effective power consumption prediction model. Firstly, extracting hardware characteristic indexes related to power consumption in a CPU, a memory, a disk and a network of a server; then, three different types of application programs, namely a CPU intensive type application program, an I/O intensive type application program and a network intensive type application program are operated, and the hardware characteristic index value and the server power consumption measurement value are recorded; then, analyzing the contribution degree of each characteristic index to the power consumption of the server by using a principal component analysis method, thereby determining the characteristic indexes participating in power consumption modeling; and finally, constructing a relation model between the determined characteristics and the power consumption through regression analysis, and completing the modeling of the power consumption of the server. Compared with the prior art, the method makes full use of the multi-element fine granularity characteristic related to the power consumption of the server, and the regression analysis modeling method has the advantages of simplicity, high efficiency and high prediction accuracy.
Description
Technical Field
The invention relates to a server power consumption modeling method based on multivariate fine-grained characteristic regression analysis, and belongs to the technical field of green computing.
Background
The rapid development of information technology has brought a huge demand for hardware infrastructure, in which servers are the basic hardware form that provides computing power for information applications. However, with the continuous enhancement of the individual performance of the servers and the continuous increase of the number of the servers, the power consumption of the servers and the clusters thereof becomes more and more prominent, which not only brings high power expenditure, but also causes environmental problems such as increased carbon emission. In order to monitor and manage the power consumption of the server, researchers respectively propose a power consumption optimization method from hardware and software layers, which comprises the following steps: dynamic power management, energy efficiency aware task scheduling, and the like. The power consumption optimization method generally needs to master the power consumption characteristics of the servers, and particularly in a cluster formed by a plurality of servers, how to perform global task scheduling according to the power consumption characteristics of different servers becomes an effective means for reducing the overall power consumption of the cluster. Therefore, the server power consumption modeling can not only describe the basic relation between the power input and the computational power output, but also serve as a solid foundation of an upper-layer power consumption optimization method.
The existing server power consumption modeling method generally selects a hardware counter capable of representing the use degree of hardware as an independent variable, a benchmark power consumption test set program is operated on a server and externally connected with a power measuring instrument, a measured power consumption value is used as a dependent variable, and a fitting relation between the independent variable and the dependent variable is obtained by a data analysis method and is used as a power consumption model of the server. A typical power consumption testing method is, for example, a testing tool provided by SPEC, which runs Java operation instructions on a server, enters different CPU usage states in fixed steps, and records corresponding power consumption values, so as to obtain a server power consumption model based on CPU usage. However, the rule of the power consumption change of the server is described by adopting a single hardware characteristic, the granularity is coarse, large errors are easy to generate, and the accuracy of a power consumption model is not favorably improved. Therefore, how to construct a more accurate server power consumption model through richer hardware characteristic indexes is an urgent problem to be solved.
Patent publication (publication) No. CN113961413A, namely Server Power consumption test method and apparatus, provides a server power consumption test method and apparatus, wherein model loading parameters during pressure test are calculated according to hardware information and resource use information of a server to be tested, server resources are pressurized according to the parameters, resource utilization rates of at least one of total bandwidths of a server CPU, a disk I/O and a network I/O are monitored in real time during pressure loading, and finally a power consumption model of the server is constructed by using the resource utilization rates and corresponding power consumption measurement values. Although the method considers the relation between the use condition of other resources except the CPU and the power consumption change of the server, the selected hardware features are still single, the integral working condition of the server is difficult to represent, and the problem of low accuracy of a power consumption model in the SPEC power consumption test also exists.
Patent publication (publication) No. CN112269715a, "server power consumption testing method and related device" provides a power consumption testing method and related device for configuring a non-volatile memory NVM virtualization platform server, which performs a pressure test on a server under test and an under-jurisdiction NVM thereof through a preset power consumption testing script, monitors the memory bandwidth thereof, and calculates the association relationship between the server power consumption and the NVM bandwidth as a power consumption model through a monitoring result. The method has certain limitation in application scenes, adopts a single index based on the memory bandwidth, and also has the problem of inaccuracy of a power consumption model.
The final objective of the above patent is to obtain a power consumption model of the server by observing the relationship between the hardware counter and the measured value of the server power consumption, thereby realizing the evaluation and prediction of the server power consumption. The prior art has the following problems: (1) The observed indexes of the hardware counter are few, and the complex operation working condition of the server is not favorably described; (2) The type of the application program of the benchmark test set is single, and different power consumption intervals of the server are difficult to cover; (3) The characteristic indexes adopted for constructing the power consumption model are relatively fixed, and the difference among different servers is not considered.
Disclosure of Invention
In order to solve the problems, the invention provides a server power consumption modeling method based on multivariate fine-grained feature regression analysis.
Firstly, extracting hardware characteristic indexes related to power consumption in a CPU, a memory, a disk and a network of a server; then, three different types of application programs, namely a CPU intensive type application program, an I/O intensive type application program and a network intensive type application program are operated, and the hardware characteristic index value and the server power consumption measurement value are recorded; then, analyzing the contribution degree of each characteristic index to the power consumption of the server by using a principal component analysis method, thereby determining the characteristic indexes participating in power consumption modeling; and finally, constructing a relation model between the determined characteristics and the power consumption through regression analysis, and completing the modeling of the power consumption of the server. Compared with the prior art, the method makes full use of the multi-element fine granularity characteristic related to the power consumption of the server, and the regression analysis modeling method has the advantages of simplicity, high efficiency and high prediction accuracy.
In order to achieve the purpose, the invention adopts the technical scheme that: a server power consumption modeling method based on multivariate fine-grained feature regression analysis comprises the following steps:
the invention discloses a server power consumption modeling method based on multivariate fine-grained characteristic regression analysis, which comprises the following steps:
step (1) feature collection: extracting and counting the characteristics related to the use degree of the hardware resources in the server to form a power consumption modeling candidate characteristic set;
step (2) pressure testing: the server runs three different types of stress test application programs, namely a CPU intensive type, an I/O intensive type and a network intensive type, and records the value of the collected characteristics in the step (1) and the measured value of the power consumption of the server;
and (3) feature screening: analyzing the contribution rate of each characteristic to the power consumption of the server by using a principal component analysis method, and screening the characteristics according to the contribution rate;
step (4), model training: and (4) constructing a relational model of the selected features and the server power consumption by adopting a regression analysis method based on the features selected in the step (3) and the measured values of the server power consumption, and completing modeling of the server power consumption.
The invention is further improved in that:
the main hardware for the server in the step (1) includes: the method comprises the steps that a CPU, a memory, a disk and a network select characteristic indexes related to power consumption to form a power consumption modeling candidate hardware characteristic index set.
The invention further improves that:
step (21) recording the power consumption of the server when the server is idle, and obtaining the maximum power consumption of the server through pressure test as the interval range of the power consumption value of the server;
and (22) running three types of application programs, namely a CPU intensive type application program, an I/O intensive type application program and a network intensive type application program, enabling the application programs to cover different working states and different power consumption values of the server as much as possible, and recording the characteristic values of the characteristic index set to be selected and the power consumption measured values corresponding to the server.
The invention further improves that: the step (3) comprises the following steps:
step (31) performing principal component analysis on the collected characteristic values and power consumption data to obtain the contribution rate of each characteristic to power consumption;
and (32) selecting a certain number of characteristic indexes as characteristics of power consumption modeling from high to low according to the obtained power consumption characteristic contribution rate, and realizing characteristic dimension reduction.
The invention further improves that:
the step (4) comprises the following steps:
step (41) according to the application scene of the power consumption model, comprehensively considering model precision and prediction efficiency, and selecting a proper regression model;
and (42) fitting and generating a power consumption model function of the server by using the characteristic values and the power consumption measured values collected in the step (3).
The invention provides a server power consumption modeling method based on multivariate fine-grained feature regression analysis, which has the following beneficial effects:
(1) And describing the operation condition of the server by adopting a rich characteristic set. Different hardware of the server usually has related hardware counter indexes to represent the use degree and the like, but the single index is difficult to describe the whole operation state of the server. By adopting the abundant hardware index feature set, the current working condition of the server can be described from multiple dimensions, and the change relation between the service state and the power consumption of the server can be more accurately represented.
(2) And covering different power consumption intervals of the server by using the multi-type benchmark test program. The server is subjected to pressure testing by adopting CPU-intensive, I/O-intensive and network-intensive benchmark test programs, on one hand, the application program is closer to a real scene, and on the other hand, the server can cover more power consumption intervals by adopting multi-type test programs, so that more real and richer training data are obtained, and the accuracy of a server power consumption model is favorably improved.
(3) There is flexibility in the hardware characterization indicators that characterize power consumption changes. The method does not adopt a preset fixed characteristic index set, but adopts a mode of 'primary selection + screening' to determine a characteristic set of power consumption modeling aiming at a server to be tested, and the screening is based on the correlation degree of related characteristics and power consumption change, thereby effectively ensuring the pertinence and reliability of the characteristics participating in the power consumption modeling in theory.
Drawings
FIG. 1 is a general schematic diagram of a server power consumption modeling method;
fig. 2 is a schematic diagram of basic steps of modeling server power consumption.
Detailed Description
The present invention will be further illustrated with reference to the accompanying drawings and specific embodiments, which are to be understood as merely illustrative of the invention and not as limiting the scope of the invention. It should be noted that the terms "front," "back," "left," "right," "upper" and "lower" used in the following description refer to directions in the drawings, and the terms "inner" and "outer" refer to directions toward and away from, respectively, the geometric center of a particular component.
The invention relates to a server power consumption modeling method based on multivariate fine-grained characteristic regression analysis, which needs to build a test platform and comprises the following steps: the server to be tested, the power analyzer and the benchmark test set are shown in fig. 1.
The server to be tested is a power consumption modeling object of the method; the power analyzer is used for measuring the actual power consumption of the server and externally connected to the server to be measured according to the relevant electrical specifications of power measurement; the benchmark set contains CPU-intensive, I/O-intensive and network-intensive applications that can cover the server power consumption interval as described in the present invention. By running a benchmark test set test, recording the index value of the server hardware and the corresponding power consumption measured value, a plurality of groups of data can be obtained, and then the data are processed and trained by the method of the invention, and finally the server power consumption model of the multiple regression is obtained.
The specific steps are shown in fig. 2, and include four steps of feature collection, stress testing, feature screening and model training.
TABLE 1 Server subsystem Power consumption characteristics index
And 2, the pressure testing step is responsible for covering different working states and power consumption intervals of the server as much as possible by running basic test set programs which are CPU intensive, I/O intensive and network intensive. Suppose that m features, denoted x, were collected by the previous step1,x2,…xm. It can be collected by sysstate tool in Linux operating system, it contains a group of tools for monitoring system performance and efficiency, and it can collect hundreds of kinds of data including CPU usage, hard disk and network throughput data. These data can be read locally by logging or by a third party by means of redf i sh, sys l og, etc. While recording the above-mentioned characteristic value, record the power consumption value data p of the power analyzer, can use the power analyzer with communication function, send the power data to the server or gather by the third party. Running n sets of benchmark programs will synchronously generate n sets of patterns such as (p: x)1,x2,…xm) The characteristic value data and the power consumption value data may be data time-aligned by time-stamping.
And 3, selecting a limited number of characteristics capable of representing the power consumption model most through a principal component analysis method in the characteristic screening step, wherein the specific steps are as follows:
(31) And (5) carrying out standardization processing on data. The standard deviation method is adopted to standardize the characteristic indexes of each sample: assuming that the value of j characteristics in the ith group of test sample data is aijConvert it into a standardized indexThe following equation is used to obtain:
wherein the content of the first and second substances,is the variance of the jth sample;is the mean of the jth sample;is a normalized feature variable.
(32) Correlation coefficient matrix R = (R)ij)m×mWherein r isijFor the correlation coefficient between the ith characteristic index and the jth characteristic index, the specific calculation mode is as follows:
(33) And calculating the eigenvalue and eigenvector of the matrix R. Solving a characteristic equation | λ I-R | =0 to obtain characteristic values λ ordered according to magnitude1≥λ2…≥λmNot less than 0, calculating lambdaiCorresponding feature vector u1,u2,…umWherein u isj=(u1j,u2j,…umj)TThe m new index variables consisting of the feature vectors are:
wherein, y1Is the first principal component, y2Is the second principal component, ymIs the mth principal component.
(34) Select k (k)<m) major components. Calculating the eigenvalue lambdaj(j =1,2, …, m) and the cumulative contribution rate. Principal component yiInformation contribution rate of Principal component y1,y2,…ykCumulative contribution rate ofWhen alpha iskAnd when the power consumption is close to 1, selecting the first k index variables as main components to replace the original m power consumption characteristic indexes.
Step 4, in the model training step, a multiple regression method is used for establishing the incidence relation between the screened features and the power consumption value, and the method specifically comprises the following steps:
(41) An appropriate regression model is selected. The previous step screens out k hardware indexes to form the shape (p)1:x11,x12,…x1k),(p2:x21,x22,…x2k),…, (pn:xn1,xn2,…xnk) N sets of sample data. And selecting a proper multiple regression model according to the change rule of the screened feature quantity and the power consumption value. For example, if the sample data satisfies multiple elastic model relationships, a relationship function in the form of the following may be selected as the model
Wherein, beta0,β1,…βkIs the regression coefficient, and e is the random error.
It should be noted that the regression models are various, and a plurality of candidate models can be considered in the selection stage, and the most suitable model can be selected by a model test method, which includes, but is not limited to, calculating the sum of squares of errors between the predicted values and the actual measured values of the models.
(42) And solving model parameters. And solving model related parameters such as regression coefficients, random errors and the like by using existing tools such as MATLAB and the like according to the n groups of sample data, and evaluating the fitting degree of the models.
Through the process, a server power consumption model based on multivariate fine-grained characteristic regression analysis can be obtained, and when power consumption prediction is carried out, the current relevant hardware index value of the server is substituted into the model for calculation, so that the power consumption prediction value of the server is obtained.
The technical means disclosed in the invention scheme are not limited to the technical means disclosed in the above embodiments, but also include the technical scheme formed by any combination of the above technical features.
Claims (5)
1. A server power consumption modeling method based on multivariate fine-grained feature regression analysis is characterized by comprising the following steps:
the method comprises the following steps:
step (1) feature collection: extracting and counting the characteristics related to the use degree of the hardware resources in the server to form a power consumption modeling candidate characteristic set;
step (2) pressure testing: the server runs three different types of pressure test application programs, namely a CPU intensive type, an I/O intensive type and a network intensive type, and records the value of the collected characteristics in the step (1) and the measured value of the power consumption of the server;
and (3) feature screening: analyzing the contribution rate of each characteristic to the power consumption of the server by using a principal component analysis method, and screening the characteristics according to the contribution rate; selecting a certain number of characteristic indexes as the characteristics of power consumption modeling from high to low to realize characteristic dimension reduction;
step (4), model training: and (4) constructing a relational model of the selected features and the server power consumption by adopting a regression analysis method based on the features selected in the step (3) and the measured values of the server power consumption, and completing modeling of the server power consumption.
2. The method for modeling the power consumption of the server based on the multivariate fine-grained feature regression analysis as claimed in claim 1, wherein:
the main hardware for the server in the step (1) includes: the method comprises the steps that a CPU, an internal memory, a disk and a network select characteristic indexes related to power consumption to form a power consumption modeling candidate hardware characteristic index set.
3. The method for modeling the power consumption of the server based on the multivariate fine-grained feature regression analysis as claimed in claim 1, wherein: the step (2) comprises the following steps
Step (21) recording the power consumption of the server when the server is idle, and obtaining the maximum power consumption of the server through pressure test as the interval range of the power consumption value of the server;
and (22) running three types of application programs, namely a CPU intensive type application program, an I/O intensive type application program and a network intensive type application program, enabling the application programs to cover different working states and different power consumption values of the server as much as possible, and recording the characteristic values of the characteristic index set to be selected and the power consumption measured values corresponding to the server.
4. The method for modeling the power consumption of the server based on the multivariate fine-grained feature regression analysis as claimed in claim 1, wherein:
the step (3) comprises the following specific steps:
step (31) standardizing the data; the standard deviation method is adopted to standardize the characteristic indexes of each sample: assuming that the value of j characteristics in the ith group of test sample data is aijConvert it into a standardized indexThe following equation is used to obtain:
wherein the content of the first and second substances,is the variance of the jth sample;is the mean of the jth sample;is a normalized feature variable;
step (32) correlation coefficient matrix R = (R)ij)m×mWherein r isijFor the correlation coefficient of the ith characteristic index and the jth characteristic index, the specific calculation mode is as follows:
step (33) calculating eigenvalues and eigenvectors of the matrix R; solving a characteristic equation | λ I-R | =0 to obtain characteristic values λ ordered according to magnitude1≥λ2…≥λmNot less than 0, calculating lambdaiCorresponding feature vector u1,u2,…umWherein u isj=(u1j,u2j,…umj)TThe m new index variables consisting of the feature vectors are:
wherein, y1Is the first principal component, y2Is the second principal component, ymIs the mth principal component;
step (34) selecting k (k < m) principal components; calculating the eigenvalue lambdaj(j =1,2, …, m) information contribution rate and cumulative contribution rate; principal component yiInformation contribution rate ofPrincipal component y1,y2,…ykCumulative contribution rate ofWhen alpha iskAnd when the power consumption is close to 1, selecting the first k index variables as main components to replace the original m power consumption characteristic indexes.
5. The method for modeling the power consumption of the server based on the multivariate fine-grained feature regression analysis as claimed in claim 1, wherein:
the step (4) comprises the following steps:
step (41) according to the application scene of the power consumption model, comprehensively considering model precision and prediction efficiency, and selecting a proper regression model;
and (42) fitting and generating a power consumption model function of the server by using the characteristic values and the power consumption measured values collected in the step (3).
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210751498.XA CN115270603A (en) | 2022-06-29 | 2022-06-29 | Server power consumption modeling method based on multivariate fine-grained feature regression analysis |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210751498.XA CN115270603A (en) | 2022-06-29 | 2022-06-29 | Server power consumption modeling method based on multivariate fine-grained feature regression analysis |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115270603A true CN115270603A (en) | 2022-11-01 |
Family
ID=83763767
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210751498.XA Pending CN115270603A (en) | 2022-06-29 | 2022-06-29 | Server power consumption modeling method based on multivariate fine-grained feature regression analysis |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115270603A (en) |
-
2022
- 2022-06-29 CN CN202210751498.XA patent/CN115270603A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108520357B (en) | Method and device for judging line loss abnormality reason and server | |
CN109409628B (en) | Acquisition terminal manufacturer evaluation method based on metering big data clustering model | |
CN106708818A (en) | Pressure testing method and system | |
CN109389145A (en) | Electric energy meter production firm evaluation method based on metering big data Clustering Model | |
CN109446812A (en) | A kind of embedded system firmware safety analytical method and system | |
CN114355094B (en) | Product reliability weak link comprehensive evaluation method and device based on multi-source information | |
CN116975567B (en) | Method, system, equipment and storage medium for testing radiation interference resistance of server | |
CN109389282A (en) | A kind of electric energy meter production firm evaluation method based on gauss hybrid models | |
CN113887866A (en) | Method and device for generating human living environment evaluation index | |
CN111177640A (en) | Data center operation and maintenance work performance evaluation system | |
CN115270603A (en) | Server power consumption modeling method based on multivariate fine-grained feature regression analysis | |
CN116244131A (en) | Server interface testing method and device, electronic equipment and medium | |
CN109389281A (en) | A kind of acquisition terminal production firm evaluation method based on gauss hybrid models | |
CN112784435B (en) | GPU real-time power modeling method based on performance event counting and temperature | |
CN113642209B (en) | Structure implantation fault response data acquisition and evaluation method based on digital twinning | |
CN115659271A (en) | Sensor abnormality detection method, model training method, system, device, and medium | |
CN113742248A (en) | Method and system for predicting organization process based on project measurement data | |
CN115165332A (en) | Integrated design method and system for built-in test and comprehensive test of equipment | |
CN113807743A (en) | Power grid dispatching automation software reliability assessment method and system | |
CN117421229A (en) | Device and method for evaluating throughput performance of HDFS (Hadoop distributed File System) | |
CN116737554B (en) | Intelligent analysis processing system and method based on big data | |
CN115577525B (en) | Design verification method and system for electric power design | |
CN116957306B (en) | User side response potential evaluation method and system based on resource collaborative interaction | |
CN116340120A (en) | Method and system for automatically constructing pressure measurement business model | |
CN106855831B (en) | System disk analysis method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |