CN110779132A - Water pump equipment operation control system of air conditioning system based on reinforcement learning - Google Patents

Water pump equipment operation control system of air conditioning system based on reinforcement learning Download PDF

Info

Publication number
CN110779132A
CN110779132A CN201911107409.2A CN201911107409A CN110779132A CN 110779132 A CN110779132 A CN 110779132A CN 201911107409 A CN201911107409 A CN 201911107409A CN 110779132 A CN110779132 A CN 110779132A
Authority
CN
China
Prior art keywords
water pump
air conditioning
pump equipment
module
control system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911107409.2A
Other languages
Chinese (zh)
Inventor
陈安琪
杨光
花静霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yaokong Technology Shanghai Co Ltd
Original Assignee
Yaokong Technology Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yaokong Technology Shanghai Co Ltd filed Critical Yaokong Technology Shanghai Co Ltd
Priority to CN201911107409.2A priority Critical patent/CN110779132A/en
Publication of CN110779132A publication Critical patent/CN110779132A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F5/00—Air-conditioning systems or apparatus not covered by F24F1/00 or F24F3/00, e.g. using solar heat or combined with household units such as an oven or water heater
    • F24F5/0003—Exclusively-fluid systems
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F11/00—Control or safety arrangements
    • F24F11/30—Control or safety arrangements for purposes related to the operation of the system, e.g. for safety or monitoring
    • F24F11/46—Improving electric energy efficiency or saving
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F11/00—Control or safety arrangements
    • F24F11/70—Control systems characterised by their outputs; Constructional details thereof
    • F24F11/80—Control systems characterised by their outputs; Constructional details thereof for controlling the temperature of the supplied air
    • F24F11/83—Control systems characterised by their outputs; Constructional details thereof for controlling the temperature of the supplied air by controlling the supply of heat-exchange fluids to heat-exchangers
    • F24F11/85—Control systems characterised by their outputs; Constructional details thereof for controlling the temperature of the supplied air by controlling the supply of heat-exchange fluids to heat-exchangers using variable-flow pumps
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F2110/00—Control inputs relating to air properties
    • F24F2110/10—Temperature
    • F24F2110/12—Temperature of the outside air
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F2110/00—Control inputs relating to air properties
    • F24F2110/30—Velocity
    • F24F2110/32—Velocity of the outside air
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F2120/00—Control inputs relating to users or occupants
    • F24F2120/10—Occupancy

Abstract

A water pump equipment operation control system of an air conditioning system based on reinforcement learning is disclosed, wherein a water pump energy consumption measuring module is used for measuring the actual energy consumption of the water pump equipment of the air conditioning system; the outdoor environment sensing measurement module is used for measuring the outdoor environment; the indoor environment sensing measurement module is used for obtaining a comprehensive feedback equivalent value; the water pump control system is used for controlling water pump equipment; the water pump operation condition feedback module is used for feeding back water pump operation condition parameters; the intelligent control module obtains the parameter data, obtains the optimal operation setting parameter point of the water pump equipment after optimization calculation, and controls the water pump equipment according to the optimal operation setting parameter point of the water pump equipment. The invention provides a water pump equipment operation control system of an air conditioning system for reinforcement learning, which has stronger adaptability to environmental change, simpler calculation process and wider development prospect.

Description

Water pump equipment operation control system of air conditioning system based on reinforcement learning
Technical Field
The invention relates to the field of air conditioner water pump control, in particular to a water pump equipment operation control system of an air conditioner system based on reinforcement learning.
Background
For many buildings, it is indispensable to construct an air conditioning system, wherein the water pump equipment in the control air conditioning system is a very important link, and the control of traditional water pump equipment can not realize the air conditioning system water pump optimal control of taking into account personnel's thermal comfort and energy consumption under indoor outdoor comprehensive environmental condition.
Disclosure of Invention
The invention aims to provide a water pump equipment operation control system of an air conditioning system for reinforcement learning aiming at the defects in the background art, the optimal operation setting parameter point of the water pump equipment is obtained through optimization calculation, the water pump equipment is controlled according to the optimal operation setting parameter point of the water pump equipment, the environment change adapting capability is stronger, the calculation process is simpler, the further optimization space on the basis is larger, and the development prospect is wider.
In order to achieve the purpose, the invention adopts the following technical scheme:
a water pump equipment operation control system of an air conditioning system based on reinforcement learning comprises a water pump energy consumption measuring module, an outdoor environment sensing measuring module, an indoor environment sensing measuring module, a water pump control system, a water pump operation condition feedback module and an intelligent control module;
the water pump energy consumption measuring module is used for measuring the actual energy consumption of water pump equipment of the air conditioning system;
the outdoor environment sensing and measuring module is used for measuring the outdoor environment to obtain the parameters of the outdoor environment;
the indoor environment sensing and measuring module is used for measuring passenger flow and indoor environment to obtain passenger flow parameters, and obtaining a comprehensive feedback equivalent value by calculating outdoor environment parameters and the passenger flow parameters;
the water pump control system is used for controlling water pump equipment;
the water pump operation condition feedback module is used for feeding back water pump operation condition parameters;
the intelligent control module is connected with and acquires parameter data uploaded by the water pump energy consumption measuring module, the outdoor environment sensing measuring module, the indoor environment sensing measuring module, the water pump control system and the water pump operation condition feedback module, optimal operation set parameter points of the water pump equipment are obtained through optimization calculation, and the water pump equipment is controlled according to the optimal operation set parameter points of the water pump equipment.
Preferably, the intelligent control module controls the water pump equipment and comprises the steps of reading environmental information, calculating an action reward value, an intelligent learning process, an action selection process and an environment interaction process;
the method comprises the steps of reading environment information, wherein the step of reading the environment information comprises the steps of obtaining outdoor environment information, indoor environment information, energy consumption information and running state information of a bottom layer controller, extracting corresponding parameters from the environment information, and obtaining an optimal indoor set point;
comprises obtaining an optimal indoor set point using the following formula;
Figure BDA0002271733200000021
K＝f(K 1，K 2，K 3，K 4) - - -formula two;
Figure BDA0002271733200000022
wherein: k 1，K 2，K 3，K 4K respectively represents intermediate coefficients for formula conversion calculation;
T outrepresenting a real-time outdoor temperature;
n represents the real-time outdoor wind speed;
V outrepresenting real-time passenger flow;
V inrepresenting real timeIndoor wind speed;
f () represents the PMV calculation function of the thermal comfort theory;
T insetrepresents an optimal indoor set point;
T inset，0standard indoor design values are expressed, and constants are specified by standard specifications.
Preferably, calculating the action reward value comprises deriving potential reward values for different actions based on the composite feedback equivalent value, as follows:
calculating potential reward values for different actions using formula four;
Figure BDA0002271733200000031
wherein:
r represents potential reward values for different actions;
α 1，α 2both represent two predicted apportioned values of energy consumption to thermal comfort ratio, the thermal comfort ratio being a known constant, (α) 1+α 2＝1)；
T inRepresenting a real-time indoor temperature;
T insetrepresents an optimal indoor set point;
e represents a real-time energy consumption value;
E setrepresenting a standard energy consumption value, a constant value specified for a standard specification;
Figure BDA0002271733200000032
representing a known temperature difference transformation function for converting the temperature difference into a fraction value, the larger the temperature difference, the lower the fraction;
Figure BDA0002271733200000033
a known energy difference transformation function is represented for converting the energy difference into a fraction value, the larger the energy difference, the lower the fraction.
Preferably, the smart learning process: considering a controlled object (water pump) as an intelligent agent, considering the cumulative influence of future series states on current selection according to a set coefficient, and integrating the instant reward obtained by the current action and the cumulative reward of the future states to obtain evaluation values of different states and different actions of the intelligent agent, wherein the evaluation values are represented by a formula V:
Q＝αR step+βQ step_next-formula five;
wherein:
q represents an action evaluation value;
R stepthe instant reward obtained by the current action is represented and obtained by a formula IV;
Q step_nextthe accumulated reward representing the future state is obtained by accumulating according to a formula four;
α denotes a conversion factor, α ═ 0.1;
β denotes a conversion factor, β is 0.5.
Preferably, the action selection process: selecting the action with the largest evaluation value as an output action signal and selecting the action with the non-largest evaluation value as a seed according to evaluation values of different states and different actions of the intelligent agent and a greedy strategy; the concrete embodiment is formula six:
Figure BDA0002271733200000041
wherein:
a represents the corresponding next action of the intelligent agent in the current state;
ε represents the greedy value set (0 < ε ≦ 1, typically taking a number greater than 0.5);
α ', α "each represent computer-generated pseudo-random numbers (0 < α' ≦ 1, 0 < α ≦ 1).
Preferably, the environment interaction process: after the intelligent agent outputs corresponding action signals, the air conditioning system interacts with the environment to obtain new environment information, and the intelligent control module repeatedly controls the water pump equipment to realize closed loop including reading the environment information, calculating an action reward value, an intelligent learning process, an action selection process and an environment interaction process.
Preferably, outdoor environment response measuring module and indoor environment response measuring module all include the sensor, outdoor environment response measuring module measures outdoor temperature, outdoor humidity and outdoor wind speed, indoor environment response measuring module measures indoor temperature and passenger flow volume, the sensor with measured data packing send to intelligent control module.
Preferably, the water pump control system controls the water pump device to include:
for the water pump equipment without a basic control link, namely the water pump equipment without the control of a controller, modifying the type of a control object to directly control the water pump equipment, and outputting a control signal transmitted to the water pump equipment as an output result;
for the existing basic control link, namely the control is carried out by the controller, the control signal is output by the bottom controller to control the water pump equipment, and the output result is the control signal output to the controller.
Preferably, the water pump operation condition feedback module feeds back water pump operation condition parameters, including:
for water pump equipment without a basic control link, namely the water pump equipment without a controller for control, the feedback parameters are water pump operation parameters;
for the existing basic control link, namely the existing controller for controlling, the running parameters of the water pump and the running parameters of the bottom layer controller are fed back.
Preferably, the system further comprises a water pump operation model module, and the water pump operation model is built by collecting historical data of enough time length.
Drawings
FIG. 1 is a system block diagram of the present invention;
fig. 2 is a flow chart of the intelligent control module controlling the water pump of the present invention.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
The orientation of the embodiment is based on the attached drawings of the specification.
The invention discloses a water pump equipment operation control system of an air conditioning system based on reinforcement learning, which is shown in figure 1 and comprises a water pump energy consumption measuring module, an outdoor environment induction measuring module, an indoor environment induction measuring module, a water pump control system, a water pump operation condition feedback module and an intelligent control module;
the water pump energy consumption measuring module is used for measuring the actual energy consumption of water pump equipment of the air conditioning system;
the outdoor environment sensing and measuring module is used for measuring the outdoor environment to obtain the parameters of the outdoor environment;
the indoor environment sensing and measuring module is used for measuring passenger flow and indoor environment to obtain passenger flow parameters, comprehensive feedback equivalent values are obtained by calculating outdoor environment parameters and passenger flow parameters, parameters reflecting the indoor actual personnel environment and parameters reflecting the outdoor environment obtained in the outdoor environment sensing and measuring module are comprehensively calculated, and according to set rules (for example, when outdoor temperature is increased, the indoor temperature set value can be properly increased under the conditions of low passenger flow and the like), an indoor personnel thermal comfort evaluation system is re-formulated to obtain the comprehensive feedback equivalent values reflecting the indoor environment; (ii) a
The water pump control system is used for controlling water pump equipment;
the water pump operation condition feedback module is used for feeding back water pump operation condition parameters;
the intelligent control module is connected with and acquires parameter data uploaded by the water pump energy consumption measuring module, the outdoor environment sensing measuring module, the indoor environment sensing measuring module, the water pump control system and the water pump operation condition feedback module, optimal operation set parameter points of the water pump equipment are obtained through optimization calculation, and the water pump equipment is controlled according to the optimal operation set parameter points of the water pump equipment.
Preferably, as shown in fig. 2, the intelligent control module controls the water pump device, including reading environmental information, calculating an action reward value, an intelligent learning process, an action selection process and an environment interaction process;
the method comprises the following steps of reading environment information, wherein the environment information comprises outdoor environment information (such as outdoor temperature and wind speed), indoor environment information (such as indoor temperature and wind speed), system operation energy consumption information (power consumption or gas consumption, unit kWh or unit), bottom layer controller operation state information (controller working state (on/off), controller control point parameter set values and the like), extracting corresponding parameters from the environment information, and obtaining an optimal indoor set point;
comprises obtaining an optimal indoor set point using the following formula;
Figure BDA0002271733200000061
K＝f(K 1，K 2，K 3，K 4) - - -formula two;
Figure BDA0002271733200000062
wherein: k 1，K 2，K 3，K 4K respectively represents intermediate coefficients for formula conversion calculation;
T outrepresenting a real-time outdoor temperature;
n represents the real-time outdoor wind speed;
V outrepresenting real-time passenger flow;
V inrepresenting real-time indoor wind speed;
f () represents the PMV calculation function of the thermal comfort theory;
T insetrepresents an optimal indoor set point;
T inset，0standard indoor design values are expressed, and constants are specified by standard specifications.
Preferably, the calculating of the action reward value includes obtaining potential reward values of different actions according to the comprehensive feedback equivalent value, firstly, according to a rule, calculating a comprehensive feedback equivalent value of an indoor environment according to indoor and outdoor environment parameters, and integrating the environment comprehensive feedback equivalent value and the operation energy consumption value according to a certain proportion (the proportion is determined according to the system use characteristics), so as to obtain potential reward values of different actions, specifically as follows:
calculating potential reward values for different actions using formula four;
wherein:
r represents potential reward values for different actions;
α 1，α 2both represent two predicted apportioned values of energy consumption to thermal comfort ratio, the thermal comfort ratio being a known constant, (α) 1+α 2＝1)；
T inRepresenting a real-time indoor temperature;
T insetrepresents an optimal indoor set point;
e represents a real-time energy consumption value;
E setrepresenting a standard energy consumption value, a constant value specified for a standard specification;
Figure BDA0002271733200000081
representing a known temperature difference transformation function for converting the temperature difference into a fraction value, the larger the temperature difference, the lower the fraction;
Figure BDA0002271733200000082
a known energy difference transformation function is represented for converting the energy difference into a fraction value, the larger the energy difference, the lower the fraction.
Preferably, the smart learning process: according to an evaluation value table under different < state, action > of the intelligent agent, according to a greedy strategy, an action with the largest evaluation value is selected as an output action signal under a certain probability, a certain exploration probability is kept, and an action with the non-largest evaluation value is selected as a seed under the other part of conditions, so that the situation that the optimal value appears at other places is prevented, and the method is represented as a formula five:
Q＝αR step+βQ step_next-formula five;
wherein:
q represents an action evaluation value;
R stepthe instant reward obtained by the current action is represented and obtained by a formula IV;
Q step_nextrepresenting future statesThe accumulated reward is obtained by accumulating according to a formula IV;
α denotes a conversion factor, α ═ 0.1;
β denotes a conversion factor, β is 0.5.
Preferably, the action selection process: selecting the action with the largest evaluation value as an output action signal and selecting the action with the non-largest evaluation value as a seed according to evaluation values of different states and different actions of the intelligent agent and a greedy strategy; the concrete embodiment is formula six:
Figure BDA0002271733200000083
wherein:
a represents the corresponding next action of the intelligent agent in the current state;
ε represents the greedy value set (0 < ε ≦ 1, typically taking a number greater than 0.5);
α ', α "each represent computer-generated pseudo-random numbers (0 < α' ≦ 1, 0 < α ≦ 1).
Preferably, the environment interaction process: after the intelligent agent outputs corresponding action signals, the air conditioning system interacts with the environment to obtain new environment information, and the intelligent control module repeatedly controls the water pump equipment to realize closed loop including reading the environment information, calculating an action reward value, an intelligent learning process, an action selection process and an environment interaction process.
Preferably, outdoor environment response measuring module and indoor environment response measuring module all include the sensor, outdoor environment response measuring module measures outdoor temperature, outdoor humidity and outdoor wind speed, indoor environment response measuring module measures indoor temperature and passenger flow volume, the sensor with measured data packing send to intelligent control module.
Preferably, the water pump control system controls the water pump device to include:
for the water pump equipment without a basic control link, namely the water pump equipment without the control of a controller, modifying the type of a control object to directly control the water pump equipment, and outputting a control signal transmitted to the water pump equipment as an output result;
for the existing basic control link, namely the control is carried out by the controller, the control signal is output by the bottom controller to control the water pump equipment, and the output result is the control signal output to the controller.
Preferably, the water pump operation condition feedback module feeds back water pump operation condition parameters, including:
for water pump equipment without a basic control link, namely the water pump equipment without a controller for control, the feedback parameters are water pump operation parameters;
for the existing basic control link, namely the existing controller for controlling, the running parameters of the water pump and the running parameters of the bottom layer controller are fed back.
Preferably, the system also comprises a water pump operation model module, the water pump operation model is built by collecting historical data of enough time length, the problem that running-in time is long when an algorithm is initialized may exist during actual application, aiming at the problem, system historical experience data can be utilized, the existing accurate water pump model is combined, a virtual environment is constructed to train the algorithm, the trained algorithm of the existing current building parameter trace can shorten the running-in time when the system is actually applied to a target building, and further optimization in time is realized. Because the training results of different water pump models are different, the advantages and disadvantages of the models can be reversely deduced and compared through the training results. The method aims to shorten the time of running-in debugging of the algorithm and the building, which is needed in the initial stage in actual application, and meanwhile, the method can also be used as a verification platform of different water pump operation models to evaluate the accuracy of different models by comparing the initial control effect.
The technical principle of the present invention is described above in connection with specific embodiments. The description is made for the purpose of illustrating the principles of the invention and should not be construed in any way as limiting the scope of the invention. Based on the explanations herein, those skilled in the art will be able to conceive of other embodiments of the present invention without inventive effort, which would fall within the scope of the present invention.

Claims (10)

1. The utility model provides a water pump equipment operation control system of air conditioning system based on reinforcement study which characterized in that: the system comprises a water pump energy consumption measuring module, an outdoor environment sensing measuring module, an indoor environment sensing measuring module, a water pump control system, a water pump operation condition feedback module and an intelligent control module;
the water pump energy consumption measuring module is used for measuring the actual energy consumption of water pump equipment of the air conditioning system;
the outdoor environment sensing and measuring module is used for measuring the outdoor environment to obtain the parameters of the outdoor environment;
the indoor environment sensing and measuring module is used for measuring passenger flow and indoor environment to obtain passenger flow parameters, and obtaining a comprehensive feedback equivalent value by calculating outdoor environment parameters and the passenger flow parameters;
the water pump control system is used for controlling water pump equipment;
the water pump operation condition feedback module is used for feeding back water pump operation condition parameters;
the intelligent control module is connected with and acquires parameter data uploaded by the water pump energy consumption measuring module, the outdoor environment sensing measuring module, the indoor environment sensing measuring module, the water pump control system and the water pump operation condition feedback module, optimal operation set parameter points of the water pump equipment are obtained through optimization calculation, and the water pump equipment is controlled according to the optimal operation set parameter points of the water pump equipment.
2. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 1, characterized in that:
the intelligent control module controls the water pump equipment and comprises the steps of reading environmental information, calculating an action reward value, an intelligent learning process, an action selection process and an environment interaction process;
the method comprises the steps of reading environment information, wherein the step of reading the environment information comprises the steps of obtaining outdoor environment information, indoor environment information, energy consumption information and running state information of a bottom layer controller, extracting corresponding parameters from the environment information, and obtaining an optimal indoor set point;
comprises obtaining an optimal indoor set point using the following formula;
Figure FDA0002271733190000011
K＝f(K 1，K 2，K 3，K 4) - - -formula two;
Figure FDA0002271733190000021
wherein: k 1，K 2，K 3，K 4K respectively represents intermediate coefficients for formula conversion calculation;
T outrepresenting a real-time outdoor temperature;
n represents the real-time outdoor wind speed;
V outrepresenting real-time passenger flow;
V inrepresenting real-time indoor wind speed;
f () represents the PMV calculation function of the thermal comfort theory;
T insetrepresents an optimal indoor set point;
T inset，0standard indoor design values are expressed, and constants are specified by standard specifications.
3. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 2, characterized in that:
calculating the action reward value includes deriving potential reward values for different actions based on the composite feedback equivalent value, as follows:
calculating potential reward values for different actions using formula four;
Figure FDA0002271733190000022
wherein:
r represents potential reward values for different actions;
α 1，α 2both represent two predicted apportioned values of energy consumption to thermal comfort ratio, the thermal comfort ratio being a known constant, (α) 1+α 2＝1)；
T inRepresenting a real-time indoor temperature;
T insetrepresents an optimal indoor set point;
e represents a real-time energy consumption value;
E setrepresenting a standard energy consumption value, a constant value specified for a standard specification;
Figure FDA0002271733190000031
representing a known temperature difference transformation function for converting the temperature difference into a fraction value, the larger the temperature difference, the lower the fraction;
Figure FDA0002271733190000032
a known energy difference transformation function is represented for converting the energy difference into a fraction value, the larger the energy difference, the lower the fraction.
4. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 3, characterized in that:
the intelligent learning process: considering a controlled object (water pump) as an intelligent agent, considering the cumulative influence of future series states on current selection according to a set coefficient, and integrating the instant reward obtained by the current action and the cumulative reward of the future states to obtain evaluation values of different states and different actions of the intelligent agent, wherein the evaluation values are represented by a formula V:
Q＝αR step+βQ step_next-formula five;
wherein:
q represents an action evaluation value;
R stepthe instant reward obtained by the current action is represented and obtained by a formula IV;
Q step_nextthe accumulated reward representing the future state is obtained by accumulating according to a formula four;
α denotes a conversion factor, α ═ 0.1;
β denotes a conversion factor, β is 0.5.
5. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 4, characterized in that:
and (3) action selection process: selecting the action with the largest evaluation value as an output action signal and selecting the action with the non-largest evaluation value as a seed according to evaluation values of different states and different actions of the intelligent agent and a greedy strategy; the concrete embodiment is formula six:
Figure FDA0002271733190000041
wherein:
a represents the corresponding next action of the intelligent agent in the current state;
ε represents the greedy value set (0 < ε ≦ 1, typically taking a number greater than 0.5);
α ', α "each represent computer-generated pseudo-random numbers (0 < α' ≦ 1, 0 < α ≦ 1).
6. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 4, characterized in that:
and (3) environment interaction process: after the intelligent agent outputs corresponding action signals, the air conditioning system interacts with the environment to obtain new environment information, and the intelligent control module repeatedly controls the water pump equipment to realize closed loop including reading the environment information, calculating an action reward value, an intelligent learning process, an action selection process and an environment interaction process.
7. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 1, characterized in that:
outdoor environment response measuring module and indoor environment response measuring module all include the sensor, outdoor environment response measuring module measures outdoor temperature, outdoor humidity and outdoor wind speed, indoor environment response measuring module measures indoor temperature and passenger flow volume, the sensor with measured data packing send to intelligent control module.
8. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 1, characterized in that:
the water pump control system controls the water pump equipment to include:
for the water pump equipment without a basic control link, namely the water pump equipment without the control of a controller, modifying the type of a control object to directly control the water pump equipment, and outputting a control signal transmitted to the water pump equipment as an output result;
for the existing basic control link, namely the control is carried out by the controller, the control signal is output by the bottom controller to control the water pump equipment, and the output result is the control signal output to the controller.
9. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 1, characterized in that:
the water pump operation condition feedback module feeds back water pump operation condition parameters, which comprises the following steps:
for water pump equipment without a basic control link, namely the water pump equipment without a controller for control, the feedback parameters are water pump operation parameters;
for the existing basic control link, namely the existing controller for controlling, the running parameters of the water pump and the running parameters of the bottom layer controller are fed back.
10. The water pump apparatus operation control system of an air conditioning system based on reinforcement learning of claim 1, characterized in that:
the system also comprises a water pump operation model module, and the water pump operation model is built by collecting historical data of enough time length.
CN201911107409.2A 2019-11-13 2019-11-13 Water pump equipment operation control system of air conditioning system based on reinforcement learning Pending CN110779132A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911107409.2A CN110779132A (en) 2019-11-13 2019-11-13 Water pump equipment operation control system of air conditioning system based on reinforcement learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911107409.2A CN110779132A (en) 2019-11-13 2019-11-13 Water pump equipment operation control system of air conditioning system based on reinforcement learning

Publications (1)

Publication Number Publication Date
CN110779132A true CN110779132A (en) 2020-02-11

Family

ID=69390844

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911107409.2A Pending CN110779132A (en) 2019-11-13 2019-11-13 Water pump equipment operation control system of air conditioning system based on reinforcement learning

Country Status (1)

Country Link
CN (1) CN110779132A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112543429A (en) * 2020-11-25 2021-03-23 北京科技大学 Configurable edge computing gateway with autonomous decision making function
CN112963946A (en) * 2021-02-26 2021-06-15 南京邮电大学 Heating, ventilating and air conditioning system control method and device for shared office area
CN115191833A (en) * 2022-07-27 2022-10-18 深圳安吉尔饮水产业集团有限公司 Control algorithm of instant heating system with dynamic learning

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003340759A (en) * 2002-05-20 2003-12-02 Sony Corp Robot device and robot control method, recording medium and program
CN101650063A (en) * 2008-08-11 2010-02-17 柯细勇 Climate compensation controller for central air conditioner and climate compensation method for central air conditioner
CN103940043A (en) * 2014-04-17 2014-07-23 美的集团股份有限公司 Comfort control method and device of air conditioner
CN104102789A (en) * 2014-07-28 2014-10-15 重庆大学 System and method for evaluating building hot and humid environment levels
US20160223218A1 (en) * 2015-01-30 2016-08-04 Schneider Electric It Corporation Automated control and parallel learning hvac apparatuses, methods and systems
CN108363356A (en) * 2017-01-26 2018-08-03 发那科株式会社 Action message learning device, action message optimize system and computer-readable medium
CN108762281A (en) * 2018-06-08 2018-11-06 哈尔滨工程大学 It is a kind of that intelligent robot decision-making technique under the embedded Real-time Water of intensified learning is associated with based on memory
CN109241552A (en) * 2018-07-12 2019-01-18 哈尔滨工程大学 A kind of underwater robot motion planning method based on multiple constraint target
CN109890176A (en) * 2019-03-01 2019-06-14 北京慧辰资道资讯股份有限公司 A kind of method and device based on artificial intelligence optimization's energy consumption of machine room efficiency
CN110378382A (en) * 2019-06-18 2019-10-25 华南师范大学 Novel quantization transaction system and its implementation based on deeply study
CN110410942A (en) * 2019-07-30 2019-11-05 上海朗绿建筑科技股份有限公司 A kind of Cooling and Heat Source machine room energy-saving optimal control method and system

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003340759A (en) * 2002-05-20 2003-12-02 Sony Corp Robot device and robot control method, recording medium and program
CN101650063A (en) * 2008-08-11 2010-02-17 柯细勇 Climate compensation controller for central air conditioner and climate compensation method for central air conditioner
CN103940043A (en) * 2014-04-17 2014-07-23 美的集团股份有限公司 Comfort control method and device of air conditioner
CN104102789A (en) * 2014-07-28 2014-10-15 重庆大学 System and method for evaluating building hot and humid environment levels
US20160223218A1 (en) * 2015-01-30 2016-08-04 Schneider Electric It Corporation Automated control and parallel learning hvac apparatuses, methods and systems
CN108363356A (en) * 2017-01-26 2018-08-03 发那科株式会社 Action message learning device, action message optimize system and computer-readable medium
CN108762281A (en) * 2018-06-08 2018-11-06 哈尔滨工程大学 It is a kind of that intelligent robot decision-making technique under the embedded Real-time Water of intensified learning is associated with based on memory
CN109241552A (en) * 2018-07-12 2019-01-18 哈尔滨工程大学 A kind of underwater robot motion planning method based on multiple constraint target
CN109890176A (en) * 2019-03-01 2019-06-14 北京慧辰资道资讯股份有限公司 A kind of method and device based on artificial intelligence optimization's energy consumption of machine room efficiency
CN110378382A (en) * 2019-06-18 2019-10-25 华南师范大学 Novel quantization transaction system and its implementation based on deeply study
CN110410942A (en) * 2019-07-30 2019-11-05 上海朗绿建筑科技股份有限公司 A kind of Cooling and Heat Source machine room energy-saving optimal control method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
陈慧岩: "《智能车辆理论与应用》", 31 July 2018, 北京理工大学出版社 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112543429A (en) * 2020-11-25 2021-03-23 北京科技大学 Configurable edge computing gateway with autonomous decision making function
CN112543429B (en) * 2020-11-25 2022-04-01 北京科技大学 Configurable edge computing gateway with autonomous decision making function
CN112963946A (en) * 2021-02-26 2021-06-15 南京邮电大学 Heating, ventilating and air conditioning system control method and device for shared office area
CN115191833A (en) * 2022-07-27 2022-10-18 深圳安吉尔饮水产业集团有限公司 Control algorithm of instant heating system with dynamic learning

Similar Documents

Publication Publication Date Title
CN110779132A (en) Water pump equipment operation control system of air conditioning system based on reinforcement learning
CN102705957B (en) Method and system for predicting hourly cooling load of central air-conditioner in office building on line
CN103912966B (en) A kind of earth source heat pump refrigeration system optimal control method
CN102003772B (en) Energy-saving optimized control method of water source heat pump
CN107860102B (en) Method and device for controlling central air conditioner
CN111829003B (en) Power plant combustion control system and control method
CN109284576B (en) Distributed electric heating load scheduling method based on measured data and modeling system thereof
CN103530818B (en) A kind of water supply network modeling method based on BRB system
CN105980786B (en) Thermic load estimating unit and air-conditioner control system
CN102779228A (en) Method and system for online prediction on cooling load of central air conditioner in marketplace buildings
CN106910144B (en) Large building time-by-time energy consumption online prediction method based on time-division actual energy consumption coefficient
CN105627506A (en) Building cooling load prediction method and device of air conditioning refrigerator
CN109523134B (en) Quantitative assessment method for time shifting capability of distributed electric heating load based on measured data and modeling system thereof
CN1201887A (en) System for monitoring expansion valve
CN109798646A (en) A kind of air quantity variable air conditioner control system and method based on big data platform
KR20200017604A (en) Energy performance prediction method and energy operation management system by using optimized physical learning model and machine learning methods
CN201476905U (en) Neural network PID temperature controlled thermocouple automatic verification system
CN111830350A (en) Energy consumption metering method and device and electric appliance
CN102122451A (en) Simulation teaching and training system for production device and method using same
CN106707999B (en) Building energy-saving system based on adaptive controller, control method and simulation
CN103279030B (en) Dynamic soft measuring modeling method and device based on Bayesian frame
KR102357589B1 (en) Device for evaluating energy performance of existing building and method thereof
CN108647817A (en) A kind of prediction technique and forecasting system of energy consumption load
CN109543235B (en) Distributed electric heating load modeling system constructed based on parameter library and modeling method thereof
CN115408930A (en) Load prediction method for subway ventilation air-conditioning system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200211