SE1651667A1 - Method and cooling system for cooling server racks in a data center - Google Patents

Abstract

METHOD AND COOLING SYSTEM FOR COOLING SERVER RACKS IN A DATA CENTRE. The present disclosure relates to a method of providing cooling of a plurality of server racks of a data centre, each server rack comprising a plurality of servers, by means of a plurality of controllers each controller being configured to control cooling of a respective server rack, wherein the method comprises: a) obtaining cooling control decision parameters relating to a lower layer responsible of cooling the server racks and relating to an upper layer responsible of optimization, supervision and coordination of the cooling of the server racks, b) determining a respective control parameter for at least some of the controllers based on the cooling control decision parameters, and c) providing a respective one of the control parameters to the corresponding controller to provide individual cooling of the server racks.(Fig. 5)

Description

15 20 25 SUMMARY In view of the above, an object of the present disclosure is to provide a method of providing cooling of a data center which solves, or at least mitigates, the problems of the prior art.

This disclosure provides new approach for data center server room ventilation and IT equipment cooling based on a multi-layer approach. A lower layer is responsible for assuring the right cooling power of each server- rack, avoiding overcooling and waste of energy. An upper layer is responsible for performing the tasks of optimization, supervision and coordination of all subsystems in the lower layer for holistic control.

There is hence according to a aspect rst aspect of the present disclosure provided a method of providing cooling of a plurality of server racks of a data center, each server rack comprising a plurality of servers, by means of a plurality of controllers each controller being conflgured to control cooling of a respective server rack, wherein the method comprises: a) obtaining cooling control decision parameters relating to a lower layer responsible of cooling the server racks and relating to an upper layer responsible of optimization, supervision and coordination of the cooling of the server racks, b) determining a respective control parameter for at least some of the controllers based on the cooling control decision parameters, and c) providing a respective one of the control parameters to the corresponding controller to provide individual cooling of the server racks.

An effect which may be obtainable thereby is that the required cooling energy may be reduced by about 20%.

According to one embodiment the cooling control decision parameters pertaining to the lower layer comprise a server rack cooling condition of each server rack.

According to one embodiment the server rack cooling condition includes at least one of temperature and air rate ow rate of the server racks. 10 15 20 25 According to one embodiment the cooling control decision parameters pertaining to the upper layer comprise server load information of each server rack.

There is according to a second aspect of the present disclosure provided a cooling system for cooling a plurality of server racks of a data center, each comprising a plurality of servers, the cooling system comprising: a plurality of controllers, each being configured to control cooling of a respective server rack, and a controller managing system configured to: obtain cooling control decision parameters relating to a lower layer responsible of cooling the server racks and relating to an upper layer responsible of optimization, supervision and coordination of the cooling of the server racks , determine a respective control parameter for at least some of the controllers based on the cooling control decision parameters, and provide a respective one of the control parameters to the corresponding controller to provide individual cooling of the server racks.

According to one embodiment the cooling control decision parameters pertaining to the lower layer comprise a server rack cooling condition of each server rack According to one embodiment the server rack cooling condition includes at least one of temperature and airfl rate of the server racks ..

According to one embodiment the cooling control decision parameters pertaining to the upper layer comprise server load information of each server rack.

Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / an / the element, apparatus, component, means, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, etc., unless explicitly stated otherwise. 10 15 20 25 BRIEF DESCRIPTION OF THE DRAWINGS The specific embodiments of the inventive concept will now be described, by way of example, With reference to the accompanying drawings, in which: Fig. 1 shows a block diagram of a multi-layer cooling control method for a data center; Fig. 2 shows a block diagram of an example of the lower layer; Fig. 3 shows a cooling system for a data center; Fig. 4 shows a block diagram of an example of an upper layer; and Fig. 5 shows a ch owchart of an example of a method of providing cooling of a data center.

DETAILED DESCRIPTION The inventive concept will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplifying embodiments are shown. The inventive concept may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art.

Like numbers refer to like elements throughout the description.

To obtain the holistic control of a data center power, IT and cooling system, it is proposed a hierarchical structure for decisions and control, where the information about the process has a different accuracy and complexity, and the amount of exchanged information and its update frequency changes according to the layer. Fig. 1 shows an example of a multi-layered control scheme for controlling the cooling of server racks, each comprising a plurality of servers.

At the upper layer, which is formed by subsystems of a controller managing system, the control strategy manages a large amount of complex information. 10 15 20 25 30 In the upper layer each subsystem of the controller managing system, as shown in the example in Fig. 4, has different frequency updates, for example, optimization and computational dynam uid dynamics (CFD) subsystems may require hours - a failure detected in an equipment can be yearly - but others subsystems are able to exchange information faster.

On the other hand, at the lower level shown in Fig. 2, the control strategy is based on the principle of increasing accuracy and decreasing intelligence and complexity, but its frequency increases to the range required for the equipment, for example, seconds for cooling units, milliseconds for power devices. The way to achieve the control implementation is to perform the control of each server rack using local controllers in the lower control layer and data center coordination in the upper control layer. This model establishes the different functional layers used for controlling a data center.

The method proposes system integration, since this ensures the control of the processes in their natural form. Then, coordination in the upper layer is achieved through the reception of information about the states of the processes (ascending path to the upper layer) and the transmission of commands to the processes (descending path to the lower layer). The exchange of information is performed between different actors in the upper layer and local controllers in the lower layer. The existence of communication devices and computer networks that ensures communication among the control equipment is a fundamental factor to achieve integration.

The lower layer is responsible for the local continuous control of cooling of each server rack. Each one is controlled by local control schemes, and they represent the direct controllers over the controlled variables. In the upper layer the tasks of optimization, supervision and coordination, for server racks to achieve a coherent operation among data centers, are provided. The operation of each server rack and its associated controller is described as the operation regions, in which the parameters and rates remain constant for an undefined time until there appears a change of the operation state. The upper layer determines the operation region for the lower layer and generates the 10 15 20 25 30 control parameters for the direct control level. Signals coming into the lower layer allow determination of the state of the process units as well as the production objectives that have been fi xed in the upper layer. At this level, local supervisors for each subsystem are defined. They determine the operation mode of each subsystem, and the changes of the parameters or local rules in the lower layer. With a superior hierarchy, a global coordinator is responsible for fixing the directives for each one of the local cooling controllers, as a function of the joint operation mode of the entire data center.

The upper layer evaluates the operation cost of the cooling system, determines the optimal amount of cooling power for each one of server rack upon the global demand for cooling power. It sends those requirements to the global system coordinator, or coordinator of cooling units, which is configured to convert them in terms of operation regions for each local controller on the server rack. The evaluation of the performance of the system is achieved by several factors such as performance, operational cost, electricity consumption and maintenance restrictions. Each operation region has a cost and an availability associated with each other, which are evaluated by other applications working over the same set of information obtained from the processes.

In view of the above a method of cooling a data center is provided. The method is based on using at least two cooling control layers, a lower layer and an upper layer. In the lower layer, electrical power demand of each server or IT load, or server load, is used as feedforward signal for estimating the cooling power required for each server rack.

There may be provided a temperature sensor network and airfl sensors configured to measure server rack cooling conditions in each server rack.

Humidity sensors do not need to be installed at each server rack. One or two humidity sensors may be installed in each server row. Information about temperatures and air flow of each server rack are used as feedback signal to achieve the right dose of power cooling on the server rack. In the lower layer a network of humidity, temperature and air pressure sensors are used for 10 15 20 25 30 controlling the humidity, temperature and air distribution on the room.

Temperature of server rack, outlet temperature or inlet temperature and airfl can be used as feedback signal, and may form part of cooling control decision parameters for the controller managing system.

The upper cooling control layer is responsible for supervising and coordinating the local controllers in the lower layer and optimizing the total cooling power and the distribution of cooling power required for each server rack.

Fig. 3 shows a block diagram of a cooling system 1, including a coordinator 9 of the cooling units or cooling equipment, which forms a subsystem of the controller managing system 3 shown in Fig. 4, and a plurality of controllers 5- 1, 5-2, ..., 5-n. The coordinator 9 is configured to receive information about cooling power of each server rack 7-1, 7-2, ..., 7-n, sensor network and cooling equipment status. Using this information the coordinator 9 is con fl gured to manage all data and send to the lower layer control parameters and configurations for each single cooling equipment, such as fan or valve, and controllers 5-1, 5-2, ..., 5- n. A similar scheme can be used to coordinate the power devices e.g. uninterruptible power source (UPS), power distribution unit batteries. Coordinators manage the communication to the lower layer at the frequency required for the lower layer units, but can also get updated information for other subsystems of the upper layer at low frequency of the range of hours e.g. CFD subsystem or optimization subsystem.

In the upper layer, tools are used which manage a large amount of complex data with low update frequency for achieving the optimal distribution of cooling power, which means air, ow, temperature and humidity distribution and patterns, and generate control parameters for the lower layer cooling control. The upper layer can use or combine many different tools, for example CFD simulations which are all less time critical compared to the operations in the lower layer. These tools optimize the global demand of cooling power of the data center taking into account complex data e.g. actual operating conditions and future expectations of IT load, electricity price, 10 15 20 25 30 power availability, and data center Operating conditions related to electrical and cooling power consumption. This data forms part of cooling control decision parameters which are used to determine the control parameters for the controllers. The upper layer provides operational information to the lower layer for the cooling units, for example, CRAC / CRAH fan speed, air temperature and humidity references for the equipment.

Fig. 4 shows an example of the upper layer. It shows how the various subsystems exchange information in the upper layer through a common communication infrastructure. For example, if the analytics subsystem predicts an imminent failure on a cooling equipment, this information can be sent to the coordinator 9 of cooling units or cooling equipment. This coordinator turns off the corresponding cooling equipment and sends a new set of parameters for the other server rack cooling controllers and cooling equipment. Coordinators exchange information with the lower layer at the frequency required for the lower layer, but it is based on information obtained from other subsystems which can exchange information at low frequency, hours in the case of CFD simulations, monthly in the case of electricity price or years when a failure is detected.

The upper layer also provides recommendations for operational information for IT load. For example, the servers that should be turned off according to the probability of being used in the near future. Placing and moving IT loads into proper locations can make the cooling infrastructure operate more efficiently and can hence result in substantial reduction in cooling power.

This methodology aims at a holistic management of the data center for avoiding waste of energy due to for example overcooling.

Functions of the upper layer include the following.

A) Power consumption data of power devices. Each power device, e.g., UPS, PDU, and battery is monitored with online power consumption.

B) IT load and load shifting strategies for cooling control. The upper layer can simulate load shifting strategies while all customer service level agreements 10 15 20 25 are ful fi lled. Furthermore, the system can simulate the arrival of new IT loads corresponding to IT load predictions which are learned from historical data.

Both the arrival of new IT jobs and the analysis of possible IT load migrations can support the cooling control of the whole data center.

C) Climate data and electrical price. This data will be used for upper layer to decide which cooling strategies to apply, free air cooling or mechanic cooling.

The upper layer will take care of energy source optimization based on electrical price and other available energy sources.

Provided cooling parameters can for example be supply cooling air temperature in rack-level, air fl ow fl owrate in rack-level, humidity in row or rack level, fl ow control through adjust the air separation angle, close / open or through ventilation ducks, server rack shutter position, CRAC / CRAH fan speed, and air temperature.

This method can be used for data center zone-level, rack-level and server-level cooling control.

With reference to Fig. 5, a ch owchart of an example of a method of cooling server racks of a data center is shown.

In a step a) cooling control decision parameters are obtained. The cooling control decision parameters are that are obtained relate to the lower layer responsible of cooling the server racks 7-1, 7-2, ..., 7-n and to the upper layer responsible of optimization, supervision and coordination of the cooling of the server racks.

The cooling control decision parameters pertaining to the lower layer may comprise a server rack cooling condition of each server rack 7-1, 7-2, ..., 7-n.

The server rack cooling condition may include at least one of temperature and air rate ow rate of the server racks 7-1, 7-2, ..., 7-n, or humidity.

The cooling control decision parameters pertaining to the upper layer may comprise server load information of each server rack 7-1, 7-2, ..., 7-n. The cooling control decision parameters may comprise additional data, for 10 15 10 example expected behavior of the server racks, concerning expected IT load / server load and so on.

In a step b) a respective control parameter is determined for at least some of the controllers 5-1, 5-2, ..., 5-n based on the cooling control decision parameters.

In a step c) a respective one of the control parameters is provided to corresponding controllers 5-1, 5-2, ..., 5-n to provide individual cooling of the server racks 7-1, 7-2, .. ., 7-n.

The controllers may for example be con fi gured to control a respective local fan of each server rack for air-cooling the server racks and / or a respective local valve of each server rack for water-cooling the server racks.

Thus, based on the server rack cooling condition and on the server load information, and / or on other cooling control decision parameters as mentioned above, the controller managing system 3 is able to determine an optimal strategy for controlling the cooling of the server racks.

The inventive concept has mainly been described above with reference to a few examples. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the inventive concept, as defined by the appended claims.

Claims (8)

10 15 20 25 11 CLAIMS 1. A method of providing cooling of a plurality of server racks (7-1, 7-2, 7- n) of a data center, each server rack (7-1, 7-2, ..., 7-n ) comprising a plurality of servers, by means of a plurality of controllers (5-1, 5-2, ..., 5-n) each controller (5-1, 5-2, ..., 5-n) being configured to control cooling of a respective server rack (7-1, 7-2, ..., 7-n), wherein the method comprises: a) obtaining cooling control decision parameters relating to a lower layer responsible of cooling the server racks (7-1, 7-2, ..., 7-n) and relating to an upper layer responsible of optimization, supervision and coordination of the cooling of the server racks (7-1, 7-2, ... , 7-n), b) determining a respective control parameter for at least some of the controllers (5-1, 5-2, ..., 5-n) based on the cooling control decision parameters, and c) providing a respective one of the control parameters to the corresponding controllers (5-1, 5-2, ..., 5-n) to provide individual cooling of the server racks (7-1, 7-2, ..., 7- n). 2. The method as claimed in claim 1, wherein the cooling control decision parameters pertaining to the lower layer comprise a server rack cooling condition of each server rack (7-1, 7-2, ..., 7-n). The method as claimed in claim 2, wherein the server rack cooling condition includes at least one of temperature and air rate ow rate of the server racks (7-1, 7-2, ..., 7-n). The method as claimed in any of claims 1-3, wherein the cooling control decision parameters pertaining to the upper layer comprise server load information of each server rack (7-1, 7-2, ..., 7-n) . 5. A cooling system (1) for cooling a plurality of server racks (7-1, 7-2, ..., 7- n) of a data center, each comprising a plurality of servers, the cooling system (1) comprising: 10 15 20 12 a plurality of controllers (5-1, 5-2, ..., 5-n), each being configured to control cooling of a respective server rack (7-1, 7-2, .. ., 7-n), and a controller managing system (3) configured to: obtain cooling control decision parameters relating to a lower layer responsible of cooling the server racks (7-1, 7-2, ..., 7-n ) and relating to an upper layer responsible of optimization, supervision and coordination of the cooling of the server racks (7-1, 7-2, ..., 7-n), determine a respective control parameter for at least some of the controllers (5-1, 5-2, ..., 5-n) based on the cooling control decision parameters, and provide a respective one of the control parameters to the corresponding controller (5-1, 5-2, .. ., 5-n) to provide individual cooling of the server racks (7-17 7_2a "'> 7_n) ° 6. The cooling system (1) as claimed in claim 5, wherein the cooling control decision parameters pertaining to the lower layer comprise a server rack cooling condition of each server rack (7-1, 7-2, ..., 7- n). 7. The cooling system (1) as claimed in claim 6, Wherein the server rack (7- 1, 7-2, ..., 7-n) cooling condition includes at least one of temperature and air rate ow rate of the server racks (7-1, 7-2, ..., 7-n). 8. The cooling system (1) as claimed in any of claims 5-7, wherein the cooling control decision parameters pertaining to the upper layer comprise server load information of each server rack (7-1, 7-2, ..., 7-n).