CN110309037A - A method for selecting features related to energy efficiency in data centers - Google Patents

Abstract

The invention proposes a method for selecting features related to energy efficiency of data centers. Aiming at the feature selection problem of data center energy efficiency, the invention adopts a feature selection method based on K-nearest neighbor classification loss function and classification interval. The energy consumption data and the corresponding PUE value, and then the PUE value is graded and classified, the corresponding classification interval is found through the sample, and the feature weight is updated and the feature weight is sorted, so as to obtain the feature selection result according to the set threshold. The method of the invention can extract the features related to the energy efficiency of the data center and process the noise data well, thereby improving the accuracy of subsequent energy efficiency prediction and effectively preventing over-learning.

Description

A method for selecting features related to energy efficiency in data centers

technical field

The invention belongs to cloud computing and machine learning, and particularly relates to a method for selecting features related to energy efficiency of a data center.

Background technique

A data center is an infrastructure that performs all-weather large-scale critical computing tasks and an important facility that supports the operation of the IT industry. Large-scale data centers with thousands of servers have proliferated as the demand for data computing, processing, and storage for large-scale cloud services by network operators and Internet companies continues to grow. Second, the cloudification of high-performance computing continues to develop with the expansion of network bandwidth, which expands the need to build large-scale computing infrastructure. As a result, data centers have become one of the key infrastructures in the rapidly growing IT industry.

In recent years, the optimization problem of energy efficiency of data centers has become critical due to the high economic efficiency and environmental relevance of data centers. First, the data center brings many economic benefits, which makes the size and number of data centers continue to grow. With the dramatic increase in electricity consumption and the rising cost of electricity, electricity bills have become a major expense in today's data centers. In some cases, the cost of electricity for a data center may be higher than the cost of the original capital investment. Secondly, the energy use of data centers will cause many environmental problems, such as large power consumption, greenhouse gas emissions from refrigeration equipment such as air conditioners, and cooling water emissions. And even if the servers in the data center are idle, they consume a lot of energy. For these reasons, energy efficiency is currently a priority in data center operations.

The most commonly used metric to measure the energy efficiency of a data center is Energy Usage Efficiency, or PUE. This metric is defined as the total energy input into the data center divided by the energy used by IT equipment. Total energy consumption includes energy used by IT equipment plus any overhead power consumed by any equipment that is not used for computing and data communications (i.e. cooling, lighting, etc.). If a data center has a PUE of 2.0, this means that for every 1 watt of energy used by the facility to supply IT equipment, other non-IT equipment will also consume 1 watt of energy. The ideal PUE is 1.0, assuming no power consumption other than IT equipment. This situation cannot be achieved in practical applications, so advanced data centers strive to have a PUE close to 1.0.

Based on the above situation, it is urgent to solve the problem of energy efficiency prediction of data centers, which has become a research hotspot at home and abroad. And one of the core tasks in energy efficiency prediction is to pick out key attributes (features) related to data center energy efficiency. Most of the current energy efficiency prediction research is based on minimal data center models, such as simple server CPU frequency or performance counter metrics, so feature selection is relatively easy. For large-scale data centers, the features are numerous and complex, and there are few studies on the selection of related features. Most of the only models are based on the black-box model of deep neural networks, which are poorly interpretable.

SUMMARY OF THE INVENTION

Purpose of the invention: In view of the insufficient selectivity of feature data in the prior art, the present invention provides a method for selecting features related to energy efficiency of data centers, which can select related features for all data centers.

Technical solution: a method for selecting features related to energy efficiency of a data center, comprising the following steps:

(1) Collect data center energy consumption data and corresponding PUE value;

(2) Grading the PUE value according to the grading standard;

(3) Randomly select a sample and find its K nearest neighbors, and calculate the classification interval corresponding to the sample at the same time;

(4) Establish a feature selection evaluation criterion based on classification loss-interval;

(5) The feature weight is updated by the evaluation criterion designed by gradient descent optimization;

(6) Sort the feature weights and obtain the feature selection result by setting the threshold.

Further, since the PUE obtained in step 1 is a continuous value, it is necessary to convert the PUE into a discrete value through a grading standard. In the step (2), the PUE value is classified according to the classification standard, and the PUE level y _i ∈ {1, 2, 3} corresponding to each data x _i is calculated according to the power utilization efficiency classification table, and x _i represents the i-th data. The n-dimensional feature vector of , where x _ij represents the jth real eigenvalue of the ith data, and its expression is as follows:

The step (3) of randomly selecting a sample and finding its K nearest neighbors, and calculating the classification interval corresponding to the sample at the same time, the specific steps are as follows:

(31) Obtain a two-dimensional binary label correspondence matrix B and a target neighbor relationship matrix T. The element b _ij ∈{0,1} in the matrix B indicates whether the PUE level y _i and y _j are the same, and the element t in the matrix T _ij ∈{0,1} indicates whether the sample x _j is the target neighbor of x _i ;

(32) The target neighbors are defined as the K-nearest neighbors of the same class with the same level as x _i PUE, where K>2;

(33) Select the sample x _i from the N samples without replacement, find the sample nearhit(x _i ) that is the nearest neighbor to the sample x i and has the same PUE level and the sample nearmiss(x _i ) that is the nearest neighbor to the sample x _i and has a different PUE level _i ), and calculate the classification interval θ _i , which is expressed as:

θ _i =|‖x _i -nearmiss(x _i )‖ ² -‖x _i -nearhit(x _i )‖ ² |.

Step (4) includes taking the loss function L _s (w, _xi ) of the sample _xi based on the feature weight w as the evaluation criterion for feature selection, which is defined as:

Among them, c is a positive number, usually obtained through cross-validation; h is the hinge loss, expressed as:

[a] ₊ =max(a,0)

Among them, the weighted Euclidean distance calculation formula is:

The step (5) includes calculating the gradient of the loss function for each feature f Finally get the n-dimensional gradient vector about all features pass Update the feature weight vector w; the loss function gradient calculation expression for feature f is as follows:

Among them, the gradient of hinge loss is defined as follows:

g(w _f )=2w _f ((x _if -x _jf ) ² -(x _if -x _pf ) ² )

The update formula of the feature weight vector w is as follows:

Finally, based on the set number of iterations, steps (3) to (5) are repeated.

The step (6) includes sorting the features according to the weight w and then setting a threshold to determine a final feature subset, where all the features in the feature subset are key features related to the energy efficiency of the data center.

Beneficial effects: Compared with the prior art, the present invention has the following significant effects: first, the present invention only needs to calculate the weights of n features and sort them, which is less computationally complex than traditional methods, and can effectively prevent Second, the K-nearest neighbor-based algorithm adopted in the present invention can better handle the noise data that may exist in the cloud data center and improve the accuracy of the data.

Description of drawings

Fig. 1 is the structural representation of the present invention;

Fig. 2 is a schematic diagram of a PUE structure;

FIG. 3 is a diagram showing the classification interval θ _i of the embodiment.

Detailed ways

In order to describe the technical solutions disclosed in the present invention in detail, further description will be given below in conjunction with the accompanying drawings and specific embodiments of the description.

First, the relevant variables involved in the present invention are introduced as follows:

Assuming that N pieces of data center energy consumption data and their PUE values have been collected, they are expressed as:

where x _i represents the n-dimensional feature vector of the i-th data, and x _ij represents the real eigenvalue of the j-th feature data of the i-th data, that is,

z _i is expressed as the PUE value corresponding to the i-th data, and the corresponding PUE level y _i ∈ {1,2,3} can be obtained according to zi _i , so a new sample can be obtained, which is expressed as:

w is the weight vector formed by the weight of each feature in the original energy efficiency dataset, where the initial value of each feature weight is 1.

The K-nearest neighbors of the same class as the _xi PUE level are defined as the target neighbors, where K>2.

The distances mentioned below in the present invention all represent Euclidean distances.

What the present invention provides is a feature selection method for all data center prediction problems, the process of which is shown in Figure 1, and the specific steps are as follows:

Step 1: Collect energy consumption data of N data centers and their corresponding PUE values.

The characteristics of the collected data center energy consumption data are as follows: total IT load of servers; total IT load of core network rooms; total number of operating process pumps; average speed of process pump inverters; total number of condensate pumps; average speed of condensate pump inverters; total number of operating cooling towers ; average set temperature of cooling tower outlet water; total number of operating chillers; total number of operating dry coolers; total number of operating chilled water injection pumps; average set temperature of chilled water injection pumps; average temperature of heat exchangers; outdoor air wet bulb temperature; outdoor air Dry bulb temperature; outdoor air enthalpy; outdoor air relative humidity; outdoor wind speed; outdoor wind direction, etc. Different data centers can capture different characteristics based on equipment or layout.

Step 2: Obtain the PUE level _yi corresponding to each piece of data _xi according to the Power Utilization Efficiency (PUE) grading table proposed by the DB11/T 1139-2014 standard. A schematic diagram of the structure of the PUE classification is shown in FIG. 2 , and the PUE classification table is shown in Table 1.

Table 1PUE grading table

level Class I Class II Class III PUE value 1＜PUE≤1.5 1.5＜PUE≤1.8 1.8＜PUE≤2.0

For example, if the PUE value corresponding to a certain piece of data in the collected data center is 1.4, it can be obtained that the PUE level corresponding to the data center is 1.

Step 3: randomly select the sample _xi and find its K nearest neighbors, and calculate the classification interval θ _i corresponding to the sample at the same time. Specifically, it is first necessary to obtain the two-dimensional binary label correspondence matrix B and the target neighbor relationship matrix T; wherein, the element b _ij ∈{0,1} in the matrix B indicates whether the PUE level y _i and y _j are the same, that is, when the sample _When x _i and sample _{x j} have the same _PUE level y _{i , y j} , that is, when the sample x _j is one of the K samples with the closest distance to the sample x _i and the corresponding PUE levels y _j , y _i are the same, t _ij =1, otherwise, t _ij =0.

Select the sample xi without replacement from the N samples, find the sample nearhit(x _i ) that is the nearest neighbor to the sample _xi and has the same PUE level and the sample nearmiss( _{xi )} that is the nearest neighbor to the sample _xi and has a different PUE level, And calculate the classification interval θ _i , as shown in Figure 3. The formula is expressed as:

θ _i =|‖x _i -nearmiss(x _i )‖ ² -‖x _i -nearhit(x _i )‖ ² |

The interval θ _i is expressed as the square of the distance between the sample nearhit(x _i ) and the sample _xi that is the closest to the sample _xi and has the same PUE level and the square of the distance to the sample _xi minus the nearmiss(x _i ) The absolute value of the squared distance from the sample _xi .

Step 4: Design a feature selection evaluation criterion based on the classification interval θ _i .

The loss function L _s (w, _xi ) of the sample _xi based on the feature weight w is used as the evaluation criterion for feature selection, which is defined as:

Among them, c is a positive number, usually obtained through cross-validation; h is the hinge loss, expressed as

[a] ₊ =max(a,0)

Among them, the weighted distance calculation formula is

The first term of the loss function represents the sum of the squares of the weighted distances between the K samples of the target neighbors of the sample _xi and the sample _xi , and the feature weight is updated to minimize the distance from the sample _xi . Weighted distance; and the second item means that for all samples _xi 's target neighbors, the square of the weighted distance from sample _xi plus the classification interval minus the K samples that are closest to sample _xi and have different PUE levels The square of the weighted distance of x _p , if the value is less than 0, it means that the sample x _p is far away from the sample x _i , which exceeds the weighted distance between the target nearest neighbor sample of the current sample x _i and the sample x _i plus the classification interval The size of , so the hinge loss is used to make the value 0; if the value is greater than 0, it means that the distance between the sample x _p and the sample x _i is relatively close, and the weight needs to be updated to minimize this item, so that the sample x _p The weighted distance from sample _xi is farther.

Through the hinge loss used in the second term of the loss function, the evaluation function is converted into a soft interval standard, which can effectively reduce the influence of outliers. At the same time, by setting the K value of the target neighbors to K>2, the noise data can be well filtered for the nearest neighbor classification.

Step 5: Update the feature weight w by optimizing the evaluation criterion by gradient descent.

Calculate the gradient of the loss function for each feature f Finally get the n-dimensional gradient vector about all features pass Update the feature weight vector w. The gradient of the loss function for feature f is calculated as follows:

Among them, the gradient of hinge loss is defined as follows:

g(w _f )=2w _f ((x _if -x _jf ) ² -(x _if -x _pf ) ² )

The update formula of the feature weight vector w is as follows:

Repeat steps 3, 4, and 5 based on the set number of iterations.

In the present invention, the computation time mainly involves the computation of matrices B, T and vector w, and their time complexity is O(N ² ), O(N ² ) and O(N ² Kn) respectively. Usually, K is a small constant. Therefore, the total time complexity of the method in the present invention is 2O(N ² )+O(N ² n)≈O(N ² n), which is better than the time complexity O(N ² n ² ) of the traditional method.

Step 6: After sorting the features according to the weight w, set a threshold to determine the final feature subset. All features in the feature subset are key features related to data center energy efficiency.

It is assumed that the features sorted according to the weight are: total IT load of servers; total number of operating chillers; total number of operating cooling towers; average set temperature of cooling tower outlet water; total number of operating process pumps; Outdoor air wet bulb temperature; outdoor air enthalpy; total IT load of core network rooms; total number of condensate pumps; average speed of condensate pump inverters; total number of running chilled water injection pumps; average set temperature of chilled water injection pumps; temperature; outdoor air dry bulb temperature; outdoor air relative humidity; outdoor wind speed; outdoor wind direction.

Assuming that the threshold is set to 14, the resulting feature subsets are: total server IT load; total number of operating chillers; total number of operating cooling towers; average set temperature of cooling tower outlet water; total number of operating process pumps; average speed of process pump inverters; Total number of running dry coolers; outdoor air wet bulb temperature; outdoor air enthalpy value; total IT load of core network rooms; total number of condensate pumps; average speed of condensate pump inverters;

The subset of features can then be used as a feature for subsequent data center energy efficiency predictions for subsequent operations.

Claims (6)

1. a method for selecting energy efficiency-related features of a data center, characterized in that: comprising the following steps: (1) Collect data center energy consumption data and corresponding PUE value; (2) Grading the PUE value according to the grading standard; (3) Randomly select a sample and find its K nearest neighbors, and calculate the classification interval corresponding to the sample at the same time; (4) Establish a feature selection evaluation criterion based on classification loss-interval; (5) The feature weight is updated by the evaluation criterion designed by gradient descent optimization; (6) Sort the feature weights and obtain the feature selection result by setting the threshold. 2. the selection method of a kind of data center energy efficiency related feature according to claim 1, it is characterized in that: in described step (2), according to the classification standard, the PUE value is graded, according to the electric energy utilization efficiency classification table, calculate each data x The PUE level corresponding to _{i i i} ∈ {1, 2, 3}, x _i represents the n-dimensional feature vector of the i-th data, where x _ij represents the j-th real eigenvalue of the i-th data, and its expression as follows: 3. The method for selecting a data center energy-efficiency-related feature according to claim 1, wherein the random selection of the sample in step (3) and its K nearest neighbors are searched, and the specific steps of calculating the classification interval corresponding to the sample simultaneously as follows: (31) Obtain a two-dimensional binary label correspondence matrix B and a target neighbor relationship matrix T. The elements b _ij ∈ {0, 1} in the matrix B indicate whether the PUE levels y _i and y _j are the same, and the element t in the matrix T _ij ∈ {0, 1} indicates whether the sample x _j is the target neighbor of x _i ; (32) The target neighbors are defined as the K-nearest neighbors of the same class with the same level as x _i PUE, where K>2; (33) Select the sample x _i from the N samples without replacement, find the sample nearhit(x _i ) that is the nearest neighbor to the sample x i and has the same PUE level and the sample nearmiss(x _i ) that is the nearest neighbor to the sample x _i and has a different PUE level _i ), and calculate the classification interval θ _i , which is expressed as: θ _i =|||x _i -nearmiss(x _i )|| ² -||x _i -nearhit(x _i )|| ² |. 4. the selection method of a kind of data center energy efficiency related feature according to claim 1, is characterized in that: step (4) comprises that sample x _i is based on the loss function L _s (w, x _i ) of feature weight w as feature selection The evaluation criteria are defined as: Among them, c is a positive number, usually obtained through cross-validation; h is the hinge loss, expressed as: [a] ₊ = max(a, 0) Among them, the weighted Euclidean distance calculation formula is: 5. The method for selecting energy-efficiency-related features of a data center according to claim 1, wherein the step (5) comprises calculating the gradient of the loss function of each feature f Finally get the n-dimensional gradient vector about all features pass Update the feature weight vector w; the loss function gradient calculation expression for feature f is as follows: Among them, the gradient of hinge loss is defined as follows: g(w _f )=2w _f ((x _if -x _jf ) ² -(x _if -x _pf ) ² ) The update formula of the feature weight vector w is as follows: Finally, based on the set number of iterations, steps (3) to (5) are repeated. 6. A method for selecting features related to energy efficiency of a data center according to claim 1, wherein the step (6) comprises sorting the features by weight w to determine the final feature subset by setting a threshold, so All the features in the described feature subset are key features related to the energy efficiency of the data center.