TWI802034B - Method for controlling air conditioner demand response - Google Patents

Abstract

A method for controlling air conditioner demand response, the method to be implemented using an air-conditioning demand response control system to adjust each temperature of N field spaces, where N is a positive integer and N≥2. The method comprises the following steps. When the air-conditioning response control system receives a demand response request command, it executes a double deep Q network machine learning model to calculate a customer baseline load corresponding to each field space according to a reward function. The air-conditioning response control system further calculates an electricity limit of each field space according to the N customer baseline loads and a total electricity limit. The double deep Q network machine learning model trains an exploratory action based on the N customer baseline loads, the N electricity limits, and N temperature feedback values. The exploratory action is related to finding a maximum reward to achieve a minimum standard of comfort.

Description

Air Conditioning Response Control Method

The invention relates to a control method, in particular to an air-conditioning response control method.

Due to the impact of global warming in recent years, in order to maintain the comfort of people in buildings, the number of air-conditioning equipment installations has continued to increase. However, extensive use of air conditioners will not only increase the burden on the grid, but also may cause unnecessary power loss if the temperature is not well controlled. Participating in demand response through air-conditioning equipment can provide system operators with effective suppression of peak loads and improve grid reliability. However, when performing demand response, excessive reduction of air-conditioning load may cause people to endure an uncomfortable environment and reduce the willingness to participate in demand response. Conversely, when the temperature control is too loose, it cannot provide enough unloading. Therefore, how to make a compromise to achieve a balance is the focus of the power management system when controlling the air-conditioning load. The shortcoming of prior art will be briefly described below:

Disadvantage 1. The use of fuzzy control theory for air-conditioning load unloading and energy saving cannot effectively consider the power consumption that needs to be unloaded in demand response, and cannot guarantee that the unloading demand will be met.

Disadvantage 2. Reducing the load of air conditioners by means of district rotation stops cannot effectively calculate the number of air conditioners in each area and the downtime. At the same time, directly shutting down the air conditioners will cause the indoor temperature to rise sharply, which cannot maintain human comfort.

Disadvantage 3: Although the rule-based air-conditioning control strategy that considers comfort can control the indoor temperature within a specific range, it cannot consider the demand response unloading amount. At the same time, as the climate environment changes, a specific range of temperature is used The limitation is the lack of adjustment flexibility and model self-correction ability.

Therefore, the object of the present invention is to provide an air-conditioning response control method that solves at least one shortcoming of the prior art and simultaneously achieves unloading requirements and comfort.

Therefore, the air-conditioning response control method of the present invention is executed by an air-conditioning response control system to adjust the temperature of N field spaces, where N is a positive integer and N≥2, and the method includes the following steps (s2)-(s4).

Step (s2) The air-conditioning response control system executes a dual deep Q network algorithm, so that the dual deep Q network algorithm performs machine learning training according to a reward function to establish a dual deep Q network machine learning model. The reward function is related to an expected dissatisfaction rate, a demand response expected power consumption and a demand response cumulative power consumption. Wherein, the definition of the demand response is related to a total power consumption limit of Taipower.

Step (s3) When the air-conditioning response control system receives a demand response request command, execute the double-depth Q-network machine learning model to calculate a reference electrical load corresponding to each field space according to the reward function , the benchmark power load is proportional to the historical average power consumption of the field space, and the double-deep Q network machine learning model calculates each field space based on the N benchmark power loads and the total power consumption limit A limited distribution of electricity.

Step (s4) The double-depth Q-network machine learning model conducts an exploration action training based on the N reference power loads, the N power-limited allocations, and the N temperature feedback values, and the exploration action is related to finding a maximum Rewarding for reaching a minimum standard of a comfort level that is inversely proportional to the expected dissatisfaction rate, the maximum reward is defined as a minimum threshold of the comfort level.

The efficacy of the present invention lies in that the air-conditioning response control method of the present invention solves the problem that the previous air-conditioning response control method cannot effectively calculate the unloading amount of the air-conditioning load, and ensures that the overall air-conditioning load can reach the demand response unloading amount signed with Taipower Company, and at the same time has Better comfort performance.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same numerals.

Referring to FIG. 1 and FIG. 2, FIG. 1 and FIG. 2 are a flow chart illustrating the steps of an embodiment of the air-conditioning response control method according to the present invention, executed by an air-conditioning response control system to adjust the temperature of N field spaces, N is a positive integer and N≥2.

In steps 100-101, the air-conditioning response control system executes an equivalent thermal model to generate a simulated temperature data of each field space, and transmits the simulated temperature data to a temperature database. The air-conditioning response control system executes a recurrent neural network algorithm, so that the recurrent neural network algorithm performs machine learning training according to the data of the temperature database to establish a recurrent neural network machine learning model.

Since it is currently difficult to obtain a large amount of temperature data in the actual field space, the equivalent thermal model is used to generate the simulated temperature data. This approach is also helpful in practical applications, because the building materials of different buildings will have their own typical thermal model parameters. After the simulated temperature data is generated through the equivalent thermal model, the recurrent neural network machine learning model can be pre-trained. The recurrent neural network machine learning model does not need to collect a large amount of environmental temperature data for training after it goes online, and the recurrent neural network machine learning model can converge to the thermal model of the actual environment faster.

Because the change of indoor temperature is time-dependent, two adjacent temperature data have a strong correlation. Therefore, the recurrent neural network machine learning model can make an accurate prediction of the indoor temperature at the next time point. The recurrent neural network machine learning model has a current indoor temperature, an outdoor temperature, and a set temperature of the air conditioner, and predicts the indoor temperature at the next time point according to the current indoor temperature, the outdoor temperature, and the set temperature of the air conditioner. temperature.

In step 102, the air conditioning response control system executes a double deep Q network (Double Deep Q Network, DDQN) algorithm, so that the double deep Q network algorithm performs machine learning training according to a reward function to establish a double deep Q network algorithm. QNet machine learning model. A single Deep Q Network (DQN) model is a deep reinforcement learning model, but there is a problem of overestimating the Q value during the learning process. By using two deep Q network models with the same architecture, the maximum Alleviate the problem of Q value overestimation, that is, the double deep Q network machine learning model.

， PPD是一預期不滿意率（Predicted Percentage of Dissatisfied，PPD），該預期不滿意率

，PMV是一預期熱感覺指標（Predicted Mean Vote，PMV），該預期熱感覺指標是採用一IOS7730標準文件第三版（發表於2005-11-15）的4.1節的式1至式4相關參數作計算，該預期不滿意率與該預期熱感覺指標關係如該IOS7730標準文件的第5節的圖1，

、

、

，將該預期不滿意率的數值映射在一二次曲線函式

以計算舒適度所獲得的獎勵，目的是引導該雙深度Q網路機器學習模型要盡可能採取一舒適度愈高的探索動作，獎勵範圍是介於－1至1。a、b、c是使

介於－1至1的常數，

是一需量反應的最後時步，

是該需量反應的累積耗電量，

是該需量反應的預期耗電量。其中，該需量反應的定義是相關於台電的一總用電量限制。 The reward function

, PPD is an expected dissatisfaction rate (Predicted Percentage of Dissatisfied, PPD), the expected dissatisfaction rate

, PMV is an expected thermal sensation index (Predicted Mean Vote, PMV), the expected thermal sensation index is the use of the third edition of the IOS7730 standard document (published on 2005-11-15) Section 4.1 related parameters of formula 1 to formula 4 For calculation, the relationship between the expected dissatisfaction rate and the expected thermal sensation index is shown in Figure 1 in Section 5 of the IOS7730 standard document,

,

,

, mapping the value of the expected dissatisfaction rate to a quadratic curve function

The reward obtained by calculating the comfort level is to guide the double-depth Q-network machine learning model to take an exploration action with a higher comfort level as much as possible, and the reward range is between -1 and 1. a, b, c are to make

a constant between -1 and 1,

is the last time step of a demand response,

is the cumulative power consumption of the demand response,

is the expected power consumption of the demand response. Wherein, the definition of the demand response is related to a total power consumption limit of Taipower.

In step 103, when the air-conditioning response control system receives a demand response request command, execute the double-depth Q-network machine learning model to calculate a reference power consumption corresponding to each field space according to the reward function load. The benchmark electricity load is proportional to the historical average electricity consumption of the field space, for example, the average electricity consumption of the previous five days when the demand response request instruction is received. The dual-depth Q-network machine learning model further calculates a limited power distribution amount for each field space according to the N reference power loads and the total power consumption limit. When the air-conditioning response control system limits the power of the N field spaces respectively according to the N power limit allocations, the recurrent neural network machine learning model respectively The corresponding N temperature feedback values are generated.

、

來進行，ε是執行一隨機探索動作的機率，

是採取可以獲得該最大獎勵的探索動作，

是一貪婪因子衰變率，

是一最小探索機率，n是一交互作用的次數，s是一場域空間的狀態，a是該探索動作。需注意的是，ε會隨著該雙深度Q網路機器學習模型的訓練次數增加而逐漸減小，目的是為了在訓練的早期增加隨機探索動作的機率，並隨著訓練次數增加，逐漸確定哪些探索動作更好後，逐漸降低隨機探索動作的機率。 In step 104 , the dual-depth Q-network machine learning model conducts an exploration action training according to the N reference electrical loads, the N power-limited allocations, and the N temperature feedback values. The exploration action is related to finding a maximum reward to achieve a minimum standard of comfort. The comfort level is inversely proportional to the expected dissatisfaction rate. For example, when people feel very hot (such as temperature range > 29 ℃) or very cold (such as temperature range < 22 ℃), the expected dissatisfaction rate will increase, on the contrary, when people feel very comfortable, very mild (25°C<temperature range<27°C), the expected dissatisfaction rate will decrease. The maximum reward is defined as a minimum threshold of the comfort level. The exploration action is using an epsilon-greedy (epsilon─Greedy) algorithm

,

to proceed, ε is the probability of performing a random exploration action,

is to take the exploration action that can get the maximum reward,

is a greedy factor decay rate,

is a minimum exploration probability, n is the number of interactions, s is the state of a field space, and a is the exploration action. It should be noted that ε will gradually decrease as the training times of the double-depth Q network machine learning model increase, the purpose is to increase the probability of random exploration actions in the early stage of training, and gradually determine After which exploration actions are better, gradually reduce the probability of random exploration actions.

In step 105, the air-conditioning response control system judges whether the current time is an execution interval of demand response.

If yes, then in step 106, the dual deep Q network machine learning model will intervene in the control of the air conditioner to perform the exploration action to find the maximum reward.

If not, then in step 107, the air-conditioning response control system generates a temperature setting value corresponding to each field space according to an input command. For example, a person takes a remote controller and inputs the temperature setting value to the air conditioner.

In step 108, the air-conditioning response control system will transmit the actual temperature data of the N field spaces to the temperature database, and return to step 101 to enable the recurrent neural network machine learning model to perform calibration training .

For example, a building has 20 rooms, and each room is equipped with an air conditioner. The demand response unloading capacity required by Taipower is to reduce the load by 20 kWh per hour, and it needs to be maintained for a total of 2 hours. In other words, the total power consumption The quantity limit is 40 kilowatt-hours, and each benchmark electric load takes the average electricity consumption of the previous five days when the demand response request instruction is received. Table 1 is the reference power load, power limit and power consumption limit of each room, as follows, wherein, limit=base power load-limit power.

Table 1. Power limit and power consumption limit of each room (degrees) Room No Base electrical load Power limit limit Room No Base electrical load Power limit limit 1 4.11 1.87 2.24 11 3.94 1.79 2.15 2 4.11 1.87 2.24 12 5.10 2.32 2.78 3 6.40 2.92 3.48 13 6.79 3.09 3.70 4 3.98 1.81 2.17 14 3.58 1.63 1.95 5 5.16 2.35 2.81 15 3.04 1.39 1.65 6 4.02 1.83 2.19 16 4.01 1.83 2.18 7 5.00 2.28 2.72 17 4.42 2.01 2.41 8 3.07 1.40 1.67 18 3.24 1.48 1.76 9 4.27 1.94 2.33 19 5.56 2.53 3.03 10 4.91 2.23 2.68 20 3.13 1.43 1.70 total 87.84 40.00 47.84

Table 2 is the power limit, power consumption limit, actual power consumption and actual power limit of each room, as follows.

Table 2. Power consumption information of each room (degrees) Room No Power limit limit Actual power consumption Actual power limit Room No Power limit limit Actual power consumption Actual power limit 1 1.87 2.24 2.03 2.08 11 1.79 2.15 1.99 1.95 2 1.87 2.24 1.80 2.30 12 2.32 2.78 2.63 2.47 3 2.92 3.48 3.44 2.96 13 3.09 3.70 3.59 3.20 4 1.81 2.17 2.07 1.91 14 1.63 1.95 1.82 1.75 5 2.35 2.81 2.58 2.58 15 1.39 1.65 1.52 1.53 6 1.83 2.19 2.07 1.95 16 1.83 2.18 2.10 1.91 7 2.28 2.72 2.24 2.76 17 2.01 2.41 2.38 2.04 8 1.40 1.67 1.53 1.55 18 1.48 1.76 1.62 1.63 9 1.94 2.33 2.20 2.07 19 2.53 3.03 2.82 2.74 10 2.23 2.68 2.39 2.51 20 1.43 1.70 1.61 1.52 total 40.00 47.84 44.43 43.41

Refer to Figure 3. Figure 3 shows that after receiving the demand response notification from Taipower, the air conditioners in each room were shut down to meet the demand response unloading capacity of 40 kilowatt-hours required by Taipower. It can be seen that the temperature rises sharply from 28°C to close to 31°C within 10 minutes, and the expected dissatisfaction rate rises sharply from close to 20% to over 60% within 10 minutes. After more than 20 minutes of shutdown, the expected dissatisfaction rate exceeds 80%. That is to say, although shutting down the air conditioner in each room can meet the demand response unloading capacity required by Taipower, the people in the room feel uncomfortable.

Refer to Figure 4. Figure 4 shows that after receiving the demand response notification from Taipower, the set temperature of the air conditioner in each room is raised to meet the demand response unloading capacity required by Taipower by 40 kWh. It can be seen that the temperature rises from 28°C to close to 29°C within 10 minutes, and the expected dissatisfaction rate rises from close to 20% to close to 25% within 10 minutes. After increasing the set temperature for more than 30 minutes, the expected dissatisfaction rate is close to 30% %. That is to say, although increasing the set temperature of the air conditioner in each room can meet the demand response unloading required by Taipower, the people in the room feel uncomfortable.

Referring to Figure 5, Figure 5 shows that after receiving the demand response notification from Taipower, the double-depth Q network machine learning model intervenes in the control of the air conditioner to perform the exploration action to find the maximum reward, so as to meet the demand response required by Taipower The unloading capacity is 40 kWh. It can be seen that the temperature remained below 28°C during the 120-minute power-limiting time, and the lowest reached 26°C, while the expected dissatisfaction rate remained at 20% during the 120-minute power-limiting time, and the lowest reached 10%. That is to say, the dual-depth Q-network machine learning model intervening in the control of air conditioners can meet the demand response unloading required by Taipower, and at the same time, people in the room feel comfortable.

Table 3 is the average expected dissatisfaction rate of the aforementioned three power reduction methods, as follows.

Table 3. The average expected dissatisfaction rate of machine learning model with shutdown, increased set temperature, and double-deep Q network Power limit mode Expected dissatisfaction rate (%) shutdown 78.63 Increase set temperature 28.30 Double Deep Q-Network Machine Learning Model 10.58

It can be seen that intervening in the control of the air conditioner through the double deep Q network machine learning model has a lower expected dissatisfaction rate than stopping the air conditioner or increasing the set temperature.

To sum up, the above-mentioned embodiment has the following advantages: Advantage 1, the double-depth Q-network machine learning model performs an exploratory action according to the N reference power loads, the N power-limited allocations and N temperature feedback values In the training of , the exploratory action is related to finding a maximum reward to achieve the minimum standard of comfort. The achievement of the effect is to solve the problem that the previous air-conditioning response control method cannot effectively calculate the unloading amount of the air-conditioning load, and ensure that the overall air-conditioning load can reach the demand response unloading amount signed with Taipower. At the same time, the air-conditioning response control method of the present invention has better comfort performance than shutting down the air-conditioning or directly heating the air-conditioning.

The second advantage is that when the number of air conditioners in the field space needs to be changed, the change can be made directly, and the double-depth Q network machine learning model and the recurrent neural network machine learning model do not need to be modified. To achieve the effect is to improve the flexibility of the model.

Advantage 3, the air-conditioning response control method of the present invention can be applied to a large number of air-conditioning special field spaces, such as schools, residential buildings, office buildings, etc., to achieve the effect, in addition to participating in the demand response announced by Taipower Corporation to earn rewards In addition, unnecessary waste of electric energy can also be avoided. Therefore really can reach the purpose of the present invention.

But the above-mentioned ones are only embodiments of the present invention, and should not limit the scope of the present invention. All simple equivalent changes and modifications made according to the patent scope of the present invention and the content of the patent specification are still within the scope of the present invention. Within the scope covered by the patent of the present invention.

100～101···························································································································································································································· 102························································································································· 103～106··································································································································································································································································································· 107 ····················· 108···································································································································································

Other features and effects of the present invention will be clearly presented in the implementation manner with reference to the drawings, wherein: Fig. 1 is a flowchart of an embodiment of the air-conditioning response control method of the present invention; Fig. 2 is a flow chart of this embodiment; Fig. 3 is a line graph of the temperature change of the air conditioner controlled by shutdown and a line diagram of the expected dissatisfaction rate of the air conditioner controlled by shutdown of the embodiment; Fig. 4 is a line graph of the temperature change of the air conditioner controlled by increasing the set temperature and a line chart of the expected dissatisfaction rate of the air conditioner controlled by increasing the set temperature; and FIG. 5 is a line graph of temperature change through the air conditioning response control method of the present invention and a line graph of expected dissatisfaction rate through the air conditioning response control method of the present invention according to the embodiment.

104～106 ······························································································································································································································································ 107 ····················· 108···································································································································································

Claims (7)

An air-conditioning response control method, executed by an air-conditioning response control system to adjust the temperature of N field spaces, N is a positive integer and N

2. The method includes: (s2) the air-conditioning response control system executes a dual-depth Q-network algorithm, so that the dual-depth Q-network algorithm performs machine learning training according to a reward function to establish a dual-depth Q-network machine learning model, the reward function is related to an expected dissatisfaction rate, an expected power consumption of a demand response and a cumulative power consumption of the demand response, wherein the definition of the demand response is related to a Taipower total power consumption limit; (s3) when the air-conditioning response control system receives a demand response request command, execute the double-depth Q-network machine learning model to calculate each of the field spaces according to the reward function One corresponds to a benchmark power load, the benchmark power load is proportional to the historical average power consumption of the field space, and the double-depth Q network machine learning model is based on the N benchmark power loads and the total power consumption Power limit, calculating each limited power distribution of these field spaces; and (s4) the double-depth Q network machine learning model is based on the N benchmark power loads, N power limited distributions and N temperatures Feedback values are used to train an exploratory action relative to finding a maximum reward to achieve a minimum level of comfort that is inversely proportional to the expected dissatisfaction rate, the maximum reward being defined as a minimum of the comfort level critical value. The air-conditioning response control method as described in Claim 1, wherein the method further includes the following steps: (s0) the air-conditioning response control system executes an equivalent thermal model to generate generate each simulated temperature data of the field spaces, and transmit the simulated temperature data to a temperature database; and (s1) the air-conditioning response control system executes a recurrent neural network algorithm, so that the recurrent neural network The roadshow algorithm performs the machine learning training according to the data of the temperature database to establish a recurrent neural network machine learning model. The air-conditioning response control method as described in claim 2, wherein the step (s3) further includes: when the air-conditioning response control system limits the power of the N field spaces respectively according to the N power-limited distribution amounts, The recurrent neural network machine learning model generates the corresponding N temperature feedback values respectively according to an actual temperature of the N field spaces. The air-conditioning response control method as described in Claim 2, wherein the method further includes the following steps: (s5) the air-conditioning response control system judges whether the current time is an execution interval of the demand response; (s6) if so, the The double deep Q network machine learning model will intervene in the control of the air conditioner to perform the exploration action to find the maximum reward; and (s7) the air conditioner response control system will send the actual temperature data of the N field spaces to the temperature data Library, and return to the step (s1) to make the recurrent neural network machine learning model perform corrective training. The air-conditioning response control method as described in claim 4, wherein the method further includes a step (s6 ' ), if the judgment result of the step (S5) is negative, the air-conditioning response control system generates a corresponding Each temperature setpoint for the field space. The air conditioner response control method as claimed in claim 1, wherein the search action in step (s4) is related to the temperature setting value of the air conditioner. The air-conditioning response control method as described in Claim 1, wherein the reward function in the step (s2) is used to guide the double-depth Q-network machine learning model to take exploration actions with higher comfort.

Images