CN116294085A - Operation method of air conditioning system of data center - Google Patents
Operation method of air conditioning system of data center Download PDFInfo
- Publication number
- CN116294085A CN116294085A CN202310373953.1A CN202310373953A CN116294085A CN 116294085 A CN116294085 A CN 116294085A CN 202310373953 A CN202310373953 A CN 202310373953A CN 116294085 A CN116294085 A CN 116294085A
- Authority
- CN
- China
- Prior art keywords
- air conditioning
- conditioning system
- data center
- temperature
- cabinet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004378 air conditioning Methods 0.000 title claims abstract description 86
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000001816 cooling Methods 0.000 claims abstract description 62
- 230000004044 response Effects 0.000 claims abstract description 60
- 230000009471 action Effects 0.000 claims abstract description 40
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 23
- 238000013178 mathematical model Methods 0.000 claims abstract description 8
- 238000011478 gradient descent method Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 30
- 230000008859 change Effects 0.000 claims description 13
- 239000003795 chemical substances by application Substances 0.000 claims description 9
- 230000009467 reduction Effects 0.000 claims description 8
- 238000005265 energy consumption Methods 0.000 claims description 5
- 238000011217 control strategy Methods 0.000 claims description 4
- 230000005611 electricity Effects 0.000 abstract description 4
- 238000013468 resource allocation Methods 0.000 abstract description 3
- 230000002787 reinforcement Effects 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 238000005057 refrigeration Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000001276 controlling effect Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000006735 deficit Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000009413 insulation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013021 overheating Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
- F24F11/00—Control or safety arrangements
- F24F11/30—Control or safety arrangements for purposes related to the operation of the system, e.g. for safety or monitoring
- F24F11/46—Improving electric energy efficiency or saving
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
- F24F11/00—Control or safety arrangements
- F24F11/50—Control or safety arrangements characterised by user interfaces or communication
- F24F11/56—Remote control
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
- F24F11/00—Control or safety arrangements
- F24F11/62—Control or safety arrangements characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
- F24F11/63—Electronic processing
- F24F11/64—Electronic processing using pre-stored data
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
- F24F11/00—Control or safety arrangements
- F24F11/70—Control systems characterised by their outputs; Constructional details thereof
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
- F24F11/00—Control or safety arrangements
- F24F11/88—Electrical aspects, e.g. circuits
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
- H05K7/00—Constructional details common to different types of electric apparatus
- H05K7/20—Modifications to facilitate cooling, ventilating, or heating
- H05K7/20709—Modifications to facilitate cooling, ventilating, or heating for server racks or cabinets; for data centers, e.g. 19-inch computer racks
- H05K7/20763—Liquid cooling without phase change
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F24—HEATING; RANGES; VENTILATING
- F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
- F24F2110/00—Control inputs relating to air properties
- F24F2110/10—Temperature
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Mechanical Engineering (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Computer Hardware Design (AREA)
- Thermal Sciences (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Air Conditioning Control Device (AREA)
Abstract
The invention provides a data center air conditioning system operation method, which comprises the steps of establishing a data center water cooling air conditioning system load response potential mathematical model according to response characteristic index analysis, obtaining the real-time state of the environment where the data center water cooling air conditioning system is located, describing the participation demand response process of the data center water cooling air conditioning system as a Markov decision process, constructing a loss function of the water cooling air conditioning system demand response based on the state space, the action space and a reward function obtained by the Markov decision process, and iteratively updating weight parameters in a current Q network in the loss function by using a gradient descent method to obtain an optimal action set so as to enable the data center water cooling air conditioning system to operate by adopting optimal actions. The invention reduces the air conditioning load of the data center in the electricity consumption peak period, optimizes the resource allocation and relieves the contradiction between the supply and the demand of electric power.
Description
Technical Field
The invention relates to the field of air conditioning system operation regulation, in particular to a data center air conditioning system operation method based on deep reinforcement learning.
Background
In recent years, reducing the energy consumption of an air conditioning system while ensuring safe and stable operation of a data center is a hot spot of research. At present, many scholars have conducted researches on the problem of optimizing the operation of an air conditioning system. Conventional Control methods, such as Rule-Based Control (RBC) methods, determine supervisory level settings for air conditioning systems, such as various temperature, frequency, and flow settings. Rules are typically static and rely on human expert knowledge to build a reasonable rule base. PID control is widely used due to the characteristics of simple control principle, low cost and the like. However, only temperature control can be realized by PID control, and optimization cannot be performed to achieve the purpose of energy saving. The intelligent control method is also a common method in the control of the air conditioning system of the data center, such as model predictive control, genetic algorithm and the like, but the method can effectively and real-time operate only under the condition of higher model accuracy. The data center water-cooling air conditioning system is an approximate system model based on mechanical cooling, electric and building thermodynamic knowledge, and an accurate system model is difficult to build, so that the method cannot obtain an ideal control effect.
The statements herein merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Disclosure of Invention
The invention aims to provide an operation method of an air conditioning system of a data center, which reduces the air conditioning load of the data center in the electricity consumption peak period, optimizes the resource allocation and relieves the contradiction between power supply and demand.
In order to achieve the above object, the present invention provides a method for operating an air conditioning system of a data center, comprising the steps of:
s1, according to response characteristic index analysis, establishing a mathematical model of the load response potential of a water-cooling air conditioning system of a data center;
s2, acquiring a real-time state of an environment where the data center water-cooling air conditioning system is located, and describing the participation demand response process of the data center water-cooling air conditioning system as a Markov decision process;
s3, constructing a loss function of the water-cooling air conditioning system demand response based on a state space, an action space and a reward function obtained by a Markov decision process;
and S4, iteratively updating weight parameters in the current Q network in the loss function by using a gradient descent method to obtain an optimal action set, so that the water-cooling air conditioning system of the data center adopts optimal action to operate.
The mathematical model of the load response potential of the water-cooled air conditioning system of the data center comprises the following components: an objective function model and a constraint condition model; the constraint condition model comprises: the system comprises a load reduction constraint model, a data center temperature change constraint model and a cabinet outlet temperature constraint model;
the objective function model is defined as:
wherein P is pot For demand response potential, kW;
punishment is carried out on the condition that the temperature of the air inlet area of the cabinet exceeds a specified range; />
Punishment for cabinet outlet temperature exceeding thermal threshold; />
The starting time and the ending time of the demand response are min;
the load shedding constraint model is defined as:
wherein P (t) is the original running power of the water-cooling air conditioning system of the data center at the moment t, namely the load baseline of the air conditioning system, kW; p (P) hvac (t) is the actual energy consumption, kW, of a data center water-cooling air conditioning system at the moment t when participating in demand response;
the data center temperature change constraint model is defined as:
heat balance expression:
Q r ＝Q IT -Q hvac
in which Q r The heat obtained in the machine room is obtained; q (Q) IT A thermal load generated for the IT device; q (Q) hvac Cooling capacity is supplied to the air conditioning system;
temperature change model of data center machine room:
ρ a ·V K ·C a ·dT(t)＝Q IT ·dt-Q hvac ·dt
wherein ρ is a ·V K ·c a dT (t) isThe heat obtained by the machine room in the dt period; ρ a Is air density; v (V) k Is the indoor volume; c a Is the specific heat capacity of air; t (T) is the temperature of the machine room in the period of T;
the cabinet outlet temperature constraint model is defined as:
in the formula, tmin max The upper limit and the lower limit of the outlet temperature range of the cabinet after the system participates in demand response regulation are set;
for the pre-cooling start time, the pre-cooling time in the demand response process is +>
If no pre-cooling strategy is used +.>
The markov decision process includes: constructing a state space, an action space and a reward function, and realizing an optimal strategy of the prefabricated cold demand response;
the state space includes:
wherein P is total The load of the water-cooled air conditioning system of the data center,
the temperature of the air inlet area of the cabinet is%>
For the temperature of the air outlet of the cabinet, T eo Water supply temperature for chilled waterDp is the pressure difference of the chilled water pump, and is influenced by the action of a control strategy, T p To advance the refrigerating time period T e For the demand response time length, S m For the time series part, including date series and small time series information, it is defined that:
wherein (1)>
Represents a time series of seven days per week, +.>
Representing a time series of days;
the action space includes: two discretization actions of chilled water supply temperature and chilled water pump pressure difference are set as the actions:
A＝[a 1 ,a 2 ...,a n ]
wherein A is an action space set, a i In (a) and (b)
With dp i Respectively providing water supply temperature of chilled water and pressure difference of the chilled water pump;
the bonus function includes:
wherein beta is 1 、β 2 Is a weight factor; p (P) pot Reducing the operation load of the water-cooling air conditioning system by kW;
a punishment function for the temperature interval exceeding the air inlet area of the specified cabinet; />
The air inlet temperature of the cabinet; t (T) low And T is up Respectively a lower limit value and an upper limit value of the air inlet temperature of the cabinet; gamma is the positive rewarding of the agent; />
Is a cabinet outlet temperature superheat threshold.
The loss function for calculating the loss value L is:
wherein,,
a t+1 for the action corresponding to the maximum Q value in the current Q network, the method comprises the steps of +.>
r t Rewards returned for the next state, α is the discount coefficient, s t+1 For the next state, a t+1 For the next action selected in the action space, θ and θ' are the weight parameters of the current Q network and the target Q network, respectively.
According to the invention, the data center water-cooling air conditioning system participates in demand response in a pre-cooling mode, deep reinforcement learning is combined with demand response, the purpose of reducing the data center air conditioning load in the electricity consumption peak period is finally realized, and an effective solution is provided for optimizing resource allocation and relieving the contradiction between power supply and demand.
Drawings
Fig. 1 is a flowchart of a method for operating a data center air conditioning system according to the present invention.
FIG. 2 is a diagram of a reinforcement learning training process jackpot according to an embodiment of the present invention.
Fig. 3 is a diagram of a cabinet outlet temperature fluctuation situation according to an embodiment of the present invention.
Fig. 4 is a diagram of a load change situation of a water-cooled air conditioning system of a data center according to an embodiment of the present invention.
FIG. 5 is a graph showing the temperature fluctuation of the outlet of a cabinet under the pre-cooling strategy according to the embodiment of the invention.
FIG. 6 is a diagram showing the load change of a water-cooled air conditioning system of a data center under a pre-cooling strategy according to an embodiment of the present invention.
FIG. 7 is a graph showing a load response potential of a water-cooled air conditioning system for a data center with different early cooling times according to an embodiment of the present invention.
Detailed Description
The following describes a preferred embodiment of the present invention with reference to fig. 1 to 7.
At present, the strategies of the air conditioning system for participating in the demand response project mainly comprise the following modes: interrupting operation, increasing set temperature, limiting fan frequency, closing a fresh air unit, controlling compressor frequency conversion and pre-cooling. However, due to the functional specificity of the building such as the data center, uninterrupted refrigeration is required throughout the year, and the enclosure structure of the data center has the characteristics of heat insulation and heat preservation, and the air conditioning system of the enclosure structure has certain thermal inertia. Therefore, the data center can be refrigerated in advance before the electricity consumption peak period comes, and the temperature of the air inlet area of the cabinet is reduced in advance, so that the pressure of the power grid load peak period can be reduced, and the operation efficiency is improved.
The invention provides a data center air conditioning system operation method based on deep learning, which can capture dynamic and nonlinear characteristics of a heat process by deep reinforcement learning under the condition of complex system modeling and shows the advantages of no model, self-adaption and online learning.
In the invention, a Double DQN algorithm in deep reinforcement learning is adopted, and the problem of overestimation of the value function is solved by using two Q networks of the Double DQN algorithm. The deep reinforcement learning has the advantages of having good capability of processing complex sequence decision-making problems, and from the aspect of algorithm efficiency, the deep reinforcement learning is continuously interacted with the environment in the initial stage of learning, and the behavior decision-making is adjusted according to feedback through a large amount of training and trial and error. However, once the model training is converged, an optimal decision can be made quickly according to the input state information, so that the real-time requirement of the algorithm is met.
As shown in fig. 1, the present invention provides a method for operating an air conditioning system of a data center, comprising the steps of:
s1, according to response characteristic index analysis, establishing a mathematical model of the load response potential of a water-cooling air conditioning system of a data center;
the objective of this patent is to maximize the load response potential of a data center water-cooled air conditioning system during demand response while minimizing the penalty of cabinet air intake area temperature and cabinet outlet temperature exceeding a prescribed range. According to the method, a mathematical model of the load response potential of the water-cooling air conditioning system of the data center, namely an objective function and constraint conditions, is established;
the mathematical model of the load response potential of the water-cooled air conditioning system of the data center comprises the following components: an objective function model and a constraint condition model, wherein the constraint condition model comprises: the system comprises a load reduction constraint model, a data center temperature change constraint model and a cabinet outlet temperature constraint model;
the objective function model is defined as:
wherein P is pot For demand response potential, kW;
punishment is carried out on the condition that the temperature of the air inlet area of the cabinet exceeds a specified range; />
Punishment for cabinet outlet temperature exceeding thermal threshold; />
The start and end times, min, of the demand response.
The load shedding constraint model is defined as:
wherein P (t) is the original running power of the water-cooling air conditioning system of the data center at the moment t, namely the load baseline of the air conditioning system, kW; p (P) hvac And (t) is the actual energy consumption, kW, of the data center water-cooling air conditioning system at the moment t when the data center water-cooling air conditioning system participates in the demand response. The load reduction constraint means that the load reduction amount of the air conditioning system of the data center is greater than or equal to the regulation potential P at each moment in the implementation process of the demand response project pot Thereby ensuring that the schedulable capacity realized by the data center air conditioning system at least reaches response potential when a demand response event occurs;
the data center temperature change constraint model is defined as:
the actual heat of the machine room consists of two parts: the cold energy generated by the air conditioning system and the heat energy generated by IT equipment in the machine room. During a certain period, the heat balance expression is as follows:
Q r ＝Q IT -Q hvac (3)
in which Q r The heat obtained in the machine room is obtained; q (Q) IT A thermal load generated for the IT device; q (Q) hvac Cooling capacity is supplied to the air conditioning system;
the air conditioning system of the data center continuously operates without stopping, and the cooling capacity Q is continuously obtained in the machine room p The load heat of IT equipment is counteracted, and the temperature of the machine room is maintained to fluctuate within a certain range, so that a data center machine room temperature change model is shown as follows:
ρ a ·V K ·c a ·dT(t)＝Q IT ·dt-Q hvac ·dt (4)
wherein ρ is a ·V K ·c a dT (t) is the heat obtained by the machine room during the dT period; ρ a Is air density; v (V) k Is the indoor volume; c a Is the specific heat capacity of air; t (T) is the temperature of the machine room in the period of T;
the cabinet outlet temperature constraint model is defined as:
in the formula, tmin max The upper limit and the lower limit of the outlet temperature range of the cabinet after the system participates in demand response regulation are set;
for the pre-cooling start time, the pre-cooling time in the demand response process is +>
If no pre-cooling strategy is used +.>
S2, acquiring a real-time state of an environment where the data center water-cooling air conditioning system is located, and describing the participation demand response process of the data center water-cooling air conditioning system as a Markov decision process;
the markov decision process includes: constructing a state space, an action space and a reward function, and realizing an optimal strategy of the prefabricated cold demand response;
the state space includes:
wherein P is total The load of the water-cooled air conditioning system of the data center,
the temperature of the air inlet area of the cabinet is%>
For the temperature of the air outlet of the cabinet, T eo For the chilled water supply temperature, dp is the pressure difference of the chilled water pump, the influence of the control strategy action is controlled, T p To advance the refrigerating time period T e For the demand response time length, S m For the time series part, including the date series and the small time series information, it can be defined as:
wherein (1)>
Represents a time series of seven days per week, +.>
Representing a time series of days;
the action space includes:
the reinforcement learning intelligent agent adopts two discretization actions of chilled water supply temperature and chilled water pump pressure difference, and the set actions are as follows:
A＝[a 1 ,a 2 …,a n ] (7)
wherein A is an action space set, a i In (a) and (b)
With dp i Respectively providing water supply temperature of chilled water and pressure difference of the chilled water pump;
the bonus function includes:
on the premise that the water-cooling air conditioning system meets the cooling load requirement of the data center, the temperature of the air inlet area of the cabinet is ensured to accord with the specified range, and the overheating of the IT equipment of the cabinet is avoided, which is the same as that of the cabinetThe maximum load reduction potential is realized, so that the rewarding function r of the problem can be obtained t ：
Wherein beta is 1 、β 2 Is a weight factor; p (P) pot Reducing the operation load of the water-cooling air conditioning system by kW;
a punishment function for the temperature interval exceeding the air inlet area of the specified cabinet; />
The air inlet temperature of the cabinet; t (T) low And T is up Respectively a lower limit value and an upper limit value of the air inlet temperature of the cabinet; gamma is the positive rewarding of the agent; />
Is a cabinet outlet temperature superheat threshold.
S3, constructing a loss function of the water-cooling air conditioning system demand response based on a state space, an action space and a reward function obtained by a Markov decision process;
loss value calculation between current Q network and target Q network:
action a t+1 Calculating a target value in a target Q network, namely:
wherein r is t Rewards returned for the next state, α is the discount coefficient, s t+1 For the next state, a t+1 For the next action selected in the action spaceθ and θ' are weight parameters of the current Q network and the target Q network, respectively;
wherein a is t+1 Is calculated as follows:
the meaning is that the action corresponding to the maximum Q value is found out in the current Q network;
from this, it can be obtained that the loss function for calculating the loss value L is:
and S4, solving a gradient of the loss value, namely updating a weight parameter theta in the current Q network by using a gradient descent method, and obtaining an optimal strategy, namely an optimal action set by an agent after continuous iteration, so that the water-cooling air conditioning system can ensure safe operation and maximally reduce the system operation load during the demand response period, and an optimal power load peak clipping effect is realized.
In the embodiment, the selected research object is a middle-sized data center in Shanghai city, and the building area of the machine room is 1500m 2 The layer height is 3m. The number of server racks is 167, the heat density of unit cabinet is 3kw, the total power of IT equipment is 500kw, and the total cooling load of the air conditioner is about 515kw. The machine room is closed, cold and hot air flows circulate and reciprocate, a special humidifying and dehumidifying device is usually arranged in the machine room, the air humidity is almost unchanged, and the air conditioner of the machine room is assumed to operate under a dry working condition. The cabinet outlet temperature threshold is set at 29 ℃. Table 1 shows the parameters of the water-cooled air conditioning system equipment in the data center.
TABLE 1
In addition, parameters related to the algorithm are set according to simulation requirements and manual experience, and optimization treatment is carried out.
Controlling the action parameter setting:
the outlet water temperature of the chilled water is controlled between 10 ℃ and 21 ℃, the discrete step length is set to be 1, and the discrete step length is divided into 12 values: [10,11,12,13,14,15,16,17,18,19,20,21]. The pressure difference of the chilled water pump is controlled between 110kPa and 150kPa, the discrete step size is set to be 10kPa, and the discrete step size is set to be 5 values: [110,120,130,140,150]. In the selection operation, the chilled water outlet temperature and the chilled pump pressure difference are combined with each other to form an operation space containing 60 elements, i.e., n=60.
Bonus function parameter settings:
according to the specification of data center design (GB 50174-2017) in China, the temperature of the air inlet area of the cabinet is 18-27 ℃, namely the upper temperature limit T up At a temperature of 27 ℃ and a lower limit T low 18 ℃;
set to 29 ℃; sigma, beta 1 ,β 2 Gamma is an adjustable super parameter, set to 100,1,1,20, all parameters have been normalized.
Super parameter setting:
the Double DQN algorithm super parameter settings are shown in table 2:
| super parameter | Parameter value |
| Learning rate alpha | 0.001 |
| Prize discount factor gamma | 0.9 |
| Experience pool capacity | 200000 |
| batch size | 256 |
| Initial search proportion ε | 0.9 |
TABLE 2
FIG. 2 is a jackpot curve during Double DQN learning. In the initial stage, the reinforcement learning agent has no training data and can only test errors continuously, and unsafe conditions that the temperature of the air inlet area of the cabinet exceeds a specified range or the temperature of the outlet of the cabinet exceeds a threshold value occur, so that the cumulative prize is small and has large fluctuation. With the increase of the training rounds, the intelligent agent gradually has good training experience to start to avoid unsafe conditions and simultaneously increase the load reduction potential, and the accumulated rewards are gradually increased at the moment. The accumulated rewards are kept stable in the later period of training, and the fact that the intelligent agent learns the optimal demand response strategy of the data center water-cooling air conditioning system for pre-refrigeration is proved, under the strategy, the data center water-cooling air conditioning system has the maximum load reduction potential in the load peak period.
Here, the results of the present invention will be verified.
In the control process of carrying out demand response verification on the water-cooling air conditioning system of the data center by using the Double DQN controller, a decision is made every 20 minutes, and 7 days are taken as one round.
Under the condition of no strategy, the period with the load rate greater than 70% is defined as a high-load rate period without considering the influence of outdoor meteorological parameters and the change of the cooling load of the data center caused by the heat carried by personnel entering and exiting, wherein the conditions of the outlet temperature of the cabinet and the load change of the water-cooling air conditioning system of the data center are shown in fig. 3 and 4.
As can be seen from fig. 3 and 4, without any optimization strategy, the average output of the water-cooled air conditioning system is 75047W during the peak load period (320 min-410 min) of the server, the high load operation duration is 90min, and the average temperature of the outlet of the cabinet is 27.9 ℃.
Assuming that the grid has a capacity deficit of 90 minutes during the peak daily load period of 320min-410min, a demand response event needs to be triggered. Based on a policy-free scene, adding a state observation quantity to realize a prefabricated cold regulating and controlling policy:
in the method, in the process of the invention,
the pre-refrigeration strategy is obtained through the study of Double DQN algorithm, and the simulation results of the strategy implementation of the outlet temperature of the lower cabinet and the air conditioner load are shown in fig. 5 and 6 respectively. Pre-refrigerating the data center in 270-320 min period by the intelligent agent reinforcement learning, wherein the pre-refrigerating time is 50min, and the average temperature of the outlet of the cabinet is reduced to 27.3 ℃; the average output of the water-cooled air conditioning system is 67890W. The cabinet outlet average temperature rose back to 28.3 ℃ within the following 320min-410min demand response time, without exceeding the temperature threshold of 29 ℃. Compared with the load curve of the water-cooled air-conditioning system of the data center in a strategy-free scene, the average output of the water-cooled air-conditioning system of the data center is reduced by 9.5% under the demand response pre-refrigeration strategy based on deep reinforcement learning, the average energy consumption of the water-cooled air-conditioning system of the data center in the load peak period is reduced, the running time of the high load rate is shortened, and the strategy has obvious peak regulation effect.
To verify the effect of the demand response strategy trained by Double DQN, the load response potential of the water-cooled air conditioner of the data center was compared under other advanced refrigeration durations, as shown in fig. 7.
As can be seen from fig. 7, the longer the pre-cooling time, the greater the potential of the water-cooled air conditioning system to reduce the load. However, if the pre-cooling time is 50 minutes and more, the load-shedding potential remains around 7 kW. Therefore, the demand response pre-cooling strategy learned by the Double DQN controller can achieve better load shedding effect.
In summary, the Double DQN controller learns a control strategy of demand response regulation and control load under the prefabricated cold scene, and the prefabricated cold strategy can reduce peak time load to a certain extent while ensuring safe and reliable operation of the data center, so as to realize the optimal power load peak clipping effect.
It should be noted that, in the embodiments of the present invention, the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the embodiments, and do not indicate or imply that the apparatus or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
While the present invention has been described in detail through the foregoing description of the preferred embodiment, it should be understood that the foregoing description is not to be considered as limiting the invention. Many modifications and substitutions of the present invention will become apparent to those of ordinary skill in the art upon reading the foregoing. Accordingly, the scope of the invention should be limited only by the attached claims.
Claims (4)
1. A method of operating a data center air conditioning system, comprising the steps of:
s1, according to response characteristic index analysis, establishing a mathematical model of the load response potential of a water-cooling air conditioning system of a data center;
s2, acquiring a real-time state of an environment where the data center water-cooling air conditioning system is located, and describing the participation demand response process of the data center water-cooling air conditioning system as a Markov decision process;
s3, constructing a loss function of the water-cooling air conditioning system demand response based on a state space, an action space and a reward function obtained by a Markov decision process;
and S4, iteratively updating weight parameters in the current Q network in the loss function by using a gradient descent method to obtain an optimal action set, so that the water-cooling air conditioning system of the data center adopts optimal action to operate.
2. The method of claim 1, wherein the mathematical model of load response potential of the data center water cooled air conditioning system comprises: an objective function model and a constraint condition model; the constraint condition model comprises: the system comprises a load reduction constraint model, a data center temperature change constraint model and a cabinet outlet temperature constraint model;
the objective function model is defined as:
wherein P is pot For demand response potential, kW;
punishment is carried out on the condition that the temperature of the air inlet area of the cabinet exceeds a specified range; />
Punishment for cabinet outlet temperature exceeding thermal threshold; />
The starting time and the ending time of the demand response are min;
the load shedding constraint model is defined as:
wherein P (t) is the original running power of the water-cooling air conditioning system of the data center at the moment t, namely the load baseline of the air conditioning system, kW; p (P) hvac (t) is the actual energy consumption, kW, of a data center water-cooling air conditioning system at the moment t when participating in demand response;
the data center temperature change constraint model is defined as:
heat balance expression:
Q r ＝Q IT -Q hvac
in which Q r The heat obtained in the machine room is obtained; q (Q) IT A thermal load generated for the IT device; q (Q) hvac Cooling capacity is supplied to the air conditioning system;
temperature change model of data center machine room:
ρ a ·V K ·c a ·dT(t)＝Q IT ·dt-Q hvac ·dt
wherein ρ is a ·V K ·c a dT (t) is the heat obtained by the machine room during the dT period; ρ a Is air density; v (V) k Is the indoor volume; c a Is the specific heat capacity of air; t (T) is the temperature of the machine room in the period of T;
the cabinet outlet temperature constraint model is defined as:
in the formula, tmin max The upper limit and the lower limit of the outlet temperature range of the cabinet after the system participates in demand response regulation are set; t (T) pcool In order to pre-prepare the cold start time,the pre-cooling time in the demand response process is as follows
If no pre-cooling strategy is used +.>
3. The method of claim 2, wherein the markov decision process comprises: constructing a state space, an action space and a reward function, and realizing an optimal strategy of the prefabricated cold demand response;
the state space includes:
wherein P is total The load of the water-cooled air conditioning system of the data center,
the temperature of the air inlet area of the cabinet is%>
For the temperature of the air outlet of the cabinet, T eo For the chilled water supply temperature, dp is the pressure difference of the chilled water pump, the influence of the control strategy action is controlled, T p To advance the refrigerating time period T e For the demand response time length, S m For the time series part, including date series and small time series information, it is defined that: />
Wherein (1)>
Represents a time series of seven days per week, +.>
Representing a time series of days;
the action space includes: two discretization actions of chilled water supply temperature and chilled water pump pressure difference are set as the actions:
A＝[a 1 ,a 2 ...,a n ]
wherein A is an action space set, a i In (a) and (b)
With dp i Respectively providing water supply temperature of chilled water and pressure difference of the chilled water pump;
the bonus function includes:
wherein beta is 1 、β 2 Is a weight factor; p (P) pot Reducing the operation load of the water-cooling air conditioning system by kW;
a punishment function for the temperature interval exceeding the air inlet area of the specified cabinet; />
The air inlet temperature of the cabinet; t (T) low And T is up Respectively a lower limit value and an upper limit value of the air inlet temperature of the cabinet; gamma is the positive rewarding of the agent; />
Is a cabinet outlet temperature superheat threshold.
4. A method of operating a data center air conditioning system as set forth in claim 3 wherein the loss function for calculating the loss value L is:
wherein,,
a t+1 for the action corresponding to the maximum Q value in the current Q network, the method comprises the steps of +.>
r t Rewards returned for the next state, α is the discount coefficient, s t+1 For the next state, a t+1 For the next action selected in the action space, θ and θ' are the weight parameters of the current Q network and the target Q network, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310373953.1A CN116294085A (en) | 2023-04-10 | 2023-04-10 | Operation method of air conditioning system of data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310373953.1A CN116294085A (en) | 2023-04-10 | 2023-04-10 | Operation method of air conditioning system of data center |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116294085A true CN116294085A (en) | 2023-06-23 |
Family
ID=86801464
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310373953.1A Pending CN116294085A (en) | 2023-04-10 | 2023-04-10 | Operation method of air conditioning system of data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116294085A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117580345A (en) * | 2024-01-19 | 2024-02-20 | 广州豪特节能环保科技股份有限公司 | Cloud computing-based centralized control method and system for indirect evaporative cooling equipment |
-
2023
- 2023-04-10 CN CN202310373953.1A patent/CN116294085A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117580345A (en) * | 2024-01-19 | 2024-02-20 | 广州豪特节能环保科技股份有限公司 | Cloud computing-based centralized control method and system for indirect evaporative cooling equipment |
| CN117580345B (en) * | 2024-01-19 | 2024-04-19 | 广州豪特节能环保科技股份有限公司 | Cloud computing-based centralized control method and system for indirect evaporative cooling equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107203239B (en) | Decoupling fuzzy control method for closed livestock and poultry breeding environment | |
| Yang et al. | Optimal control strategy for HVAC system in building energy management | |
| CN114963414B (en) | Intelligent air conditioning system regulating and controlling device based on AI data analysis | |
| CN116294085A (en) | Operation method of air conditioning system of data center | |
| CN110848895B (en) | Non-industrial air conditioner flexible load control method and system | |
| CN110940061A (en) | Central air conditioner control method and system | |
| CN114659237B (en) | Air conditioner energy efficiency supervision method based on Internet of things | |
| CN110486896B (en) | Cascade air conditioning system optimization control method based on water chilling unit energy consumption model | |
| CN113566401B (en) | Demand side load control method | |
| CN112484250A (en) | HVAC (heating ventilation and ventilation air conditioning) online monitoring system based on indoor heat source information and control method | |
| JP6291756B2 (en) | Information processing apparatus, information processing method, program, and recording medium | |
| Li et al. | Reinforcement learning-based demand response strategy for thermal energy storage air-conditioning system considering room temperature and humidity setpoints | |
| Gao et al. | An optimal control method for small-scale GSHP-integrated air-conditioning system to improve indoor thermal environment control | |
| Vakiloroaya et al. | Component-wise optimization for a commercial central cooling plant | |
| CN111928428B (en) | Control method of air conditioning system considering demand response and refrigeration system | |
| CN113757852B (en) | Multi-split air conditioning unit control method and system based on digital twinning technology | |
| Wang et al. | A Data-driven Control Method for Operating the Commercial HVAC Load as a Virtual Battery | |
| CN116697548B (en) | Be applied to purification air conditioning unit and purification air conditioning unit intelligent control system of hospital | |
| CN109190988A (en) | A kind of Demand Side Response game method for realizing the optimal collaboration of temperature control load | |
| CN114115398B (en) | Demand response control method and device for building water chilling unit | |
| JP5849879B2 (en) | Heat pump system and control method thereof | |
| CN209861437U (en) | Equipment cabinet cooling and heat dissipation system | |
| CN112665159A (en) | Load rebound quantity optimization and load regulation method and system based on demand response | |
| CN115839536B (en) | Remote response and control method for split air conditioner and water chilling unit based on electric power regulation | |
| US20220373210A1 (en) | Chiller controller for optimized efficiency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |