CN112821465B - Industrial microgrid load optimization scheduling method and system containing cogeneration - Google Patents

Industrial microgrid load optimization scheduling method and system containing cogeneration Download PDF

Info

Publication number
CN112821465B
CN112821465B CN202110022857.3A CN202110022857A CN112821465B CN 112821465 B CN112821465 B CN 112821465B CN 202110022857 A CN202110022857 A CN 202110022857A CN 112821465 B CN112821465 B CN 112821465B
Authority
CN
China
Prior art keywords
power
representing
period
heat
heat storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110022857.3A
Other languages
Chinese (zh)
Other versions
CN112821465A (en
Inventor
周开乐
周昆树
焦建玲
杨善林
殷辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei University of Technology
Original Assignee
Hefei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hefei University of Technology filed Critical Hefei University of Technology
Priority to CN202110022857.3A priority Critical patent/CN112821465B/en
Publication of CN112821465A publication Critical patent/CN112821465A/en
Application granted granted Critical
Publication of CN112821465B publication Critical patent/CN112821465B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
    • H02J3/46—Controlling of the sharing of output between the generators, converters, or transformers
    • H02J3/466—Scheduling the operation of the generators, e.g. connecting or disconnecting generators to meet a given demand
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/008—Circuit arrangements for ac mains or ac distribution networks involving trading of energy or energy transmission rights
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
    • H02J2203/20—Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P80/00—Climate change mitigation technologies for sector-wide applications
    • Y02P80/10—Efficient use of energy, e.g. using compressed air or pressurized fluid as energy carrier

Landscapes

  • Engineering & Computer Science (AREA)
  • Power Engineering (AREA)
  • Supply And Distribution Of Alternating Current (AREA)

Abstract

The invention provides an industrial micro-grid load optimization scheduling method and system comprising cogeneration, and relates to the field of industrial micro-grid load optimization scheduling. Preprocessing state parameters of each component of the industrial microgrid; constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters; converting the micro energy network load optimization scheduling model into a Markov decision process; and solving the Markov decision process by adopting a deep Q network model to obtain a micro energy network load optimization scheduling strategy. The load optimization scheduling system of the industrial micro-grid comprising the cogeneration equipment is constructed substantially, and energy consumption is saved for operation of an industrial enterprise from the perspective of actual production of the industrial enterprise; in consideration of the coupling operation characteristics of three energy flows of electricity, heat and gas under the background of cogeneration application, the demand response potential of a user side is excavated; facilitating benign interaction of multi-energy upper and lower level networks; the data-driven deep reinforcement learning method can effectively improve the efficiency and accuracy of optimized scheduling.

Description

Industrial microgrid load optimization scheduling method and system containing cogeneration
Technical Field
The invention relates to the technical field of industrial microgrid load optimization scheduling, in particular to an industrial microgrid load optimization scheduling method and system comprising cogeneration.
Background
Combined heat and power (combined heat and power) equipment can integrate two energy sources of electricity and gas, fully play the synergy and complementary action between the two, improve the utilization efficiency of the whole energy source and promote the consumption of renewable energy sources. The cogeneration system can provide heat energy while generating electricity, can realize the process of simultaneously generating electric energy and heat energy, and can save energy sources compared with the traditional separated energy supply system, so the cogeneration system is widely applied to industrial production of chemical industry, papermaking, cement, steel and the like. As shown in fig. 1, constructing a micro energy network containing cogeneration equipment is also one of the main approaches to solve the problems of unreasonable heat supply source structure, outstanding contradiction between heat and power supply and demand, low energy efficiency of heat supply source, and the like in the production process of industrial enterprises. On the other hand, different from the operation optimization of a single power supply system, due to the existence of equipment coupling and the access of various equipment and various loads, the cogeneration system needs to face a more complicated and changeable operation environment, and great challenges are brought to the intelligent optimization scheduling of the system.
The comprehensive energy system developed in the form of the micro energy network breaks through the existing mode that each traditional energy system operates independently, so that the coupling of the multi-energy flows in different forms is tighter and tighter, and the operation modes of mutual substitution and mutual supplement provide a new comprehensive demand response way for the energy demand side. The Combined Heat and Power Economic Dispatch (CHPED) is an important research content of combined heat and power equipment in the application of the micro-energy network of the industrial enterprise under the background of energy internet, and can be divided into day-ahead optimized dispatch (every 24 hours) and day-in optimized dispatch (every 1 hour or every 5-15 minutes) according to different dispatch periods. The day-ahead optimized scheduling determines the next-day operation strategy of the system by predicting the output of renewable energy sources (the power generation power of the renewable energy sources such as photovoltaic or wind turbine) and the load demand, so as to ensure the economic and reliable operation of the system. CHPED problems are generally viewed as optimization problems with one or more optimization objectives, and with a set of highly nonlinear or non-smooth constraints, which generally include energy supply and demand balance constraints, equipment operating constraints, and capacity limitation constraints. The industrial enterprise has the characteristics of large energy demand and high functional reliability in the production process, so that the design and the proposal of the industrial micro-grid optimal scheduling method and the system containing the cogeneration equipment with comprehensive demand response participation in the energy internet background have important significance.
At present, the conventional algorithms commonly used for solving the CHPED problem include an equal differential gain method, a lagrange multiplier method, a linear programming method, a dynamic programming method and the like. The algorithm has the advantages of high calculation speed and high precision, but has strong solving capability only for the optimization problem that the objective function and the constraint condition are both convex functions, has certain limitation, and cannot be applied to the load optimization scheduling problem of the micro energy network containing various complex devices. In contrast, the intelligent optimization algorithm has stronger adaptability, can solve various optimization problems, and is also widely applied to the solution of the optimal scheduling model of the power system.
However, even if the intelligent optimization algorithm is used for solving the problem of optimal scheduling of the power system, real-time optimal scheduling cannot be achieved, which affects the efficiency and accuracy of the optimization process for optimal scheduling of the power system, especially for load optimal scheduling of the industrial microgrid including cogeneration.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides an industrial micro-grid load optimization scheduling method and system comprising cogeneration, and solves the technical problem that the existing micro-energy grid load optimization scheduling model cannot perform real-time optimization scheduling.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
an industrial microgrid load optimization scheduling method comprising cogeneration comprises the following steps:
s1, preprocessing state parameters of all components of an industrial micro-grid containing cogeneration;
s2, constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters;
s3, converting the micro energy network load optimization scheduling model into a Markov decision process;
and S4, solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy containing combined heat and power generation.
Preferably, the building process of the micro energy grid load optimization scheduling model in the step S2 is based on a combined heat and power generation comprehensive energy demand response mechanism, and specifically includes an objective function:
min cost C＝C 1 +C 2 +C 3 +C 4 (1)
wherein, C 1 Representing the electricity purchase and sale cost:
Figure GDA0003762154670000031
in the formula (2)
Figure GDA0003762154670000032
And
Figure GDA0003762154670000033
respectively representing the prices of electricity purchase and electricity sale of the micro energy grid from the large power grid; p grid，t The electricity purchasing/selling quantity of the micro energy network in the t period is represented, a positive value represents that the micro energy network purchases electricity, and a negative value represents that the micro energy network sells electricity to the large power grid; t represents the whole scheduling period;
C 2 representing the gas purchase cost:
Figure GDA0003762154670000034
g in the formula (3) grid，t Representing the amount of natural gas purchased from a natural gas network by the micro energy network in the t period;
Figure GDA0003762154670000041
representing the natural gas price for the t period;
C 3 represents the depreciation cost of the energy storage device:
Figure GDA0003762154670000042
the first term in equation (4) is the battery depreciation cost, where P char，t Represents the battery charging power for a period t; p dis，t Represents the battery discharge power in the t period; because the depreciation cost of the energy storage battery is in direct proportion to the energy storage charging and discharging electric quantity, the ratio of the depreciation cost to the energy storage charging and discharging electric quantity is called depreciation coefficient and is recorded as k, and the calculation mode is
Figure GDA0003762154670000043
Figure GDA0003762154670000044
Price in the formula ess The price of the energy storage battery is shown,
Figure GDA0003762154670000045
the rated capacity of the energy storage battery is shown, and the cycle life of the battery is shown by L;
the second term is the depreciation cost of the heat storage tank, wherein h is the depreciation coefficient of the heat storage tank, and the calculation mode is
Figure GDA0003762154670000046
price tst Which represents the price of the heat storage tank,
Figure GDA0003762154670000047
the rated capacity of the heat storage tank is shown, and M represents the cycle life of the heat storage tank;
C 4 represents the equipment operation and maintenance cost:
C 4 ＝P gt，t *K gt +(|P char，t |+|P dis，t |)*K bt +P pv，t *K pv +H gb，t *K gb +H hr，t *K hr (5)
formula (5) wherein P gt，t Electrical power representing a gas turbine time period t; k bt Represents a gas turbine operating maintenance cost; (| P) char，t |+|P dis，t |) represents the charge-discharge power of the storage battery at t period; k bt Representing the running and maintenance cost of the storage battery; p pv，t Output electric power representing a period t of the photovoltaic panel; k pv Representing the operating and maintenance cost of the photovoltaic panel; h gb，t Output power representing a period t of the gas boiler; k is gb The operating and maintenance costs for the gas boiler; h hr，t Output power, K, representing a period t of the waste heat recovery device hr Representing the operating and maintenance costs of the waste heat recovery device.
Preferably, the process of constructing the micro energy grid load optimization scheduling model in step S2 further includes constraint conditions:
A. power supply and demand balance constraint:
Figure GDA0003762154670000051
p in formula (6) grid，t Large electric network and micro energy for t periodTransmission power between source networks, P pv，t Output electric power, P, representing the t period of the photovoltaic panel ess，t Represents the charging and discharging power of the battery during the period t, P ess，t ＝P dis，t -P char，t ，P gt，t Electric power, P, representing the t period of the gas turbine load，t Representing the total electrical load demand in the micro energy grid during time t;
H hr，t the output thermal power of the waste heat recovery device in a t period is represented; h gb，t Representing the output thermal power of the gas boiler during the t period; h dis，t Represents the heat release power of the heat storage tank t period; h char，t Represents the heat storage power of the heat storage tank t period; h load，t Representing the total heat load demand in the micro energy network in the time period t;
B. and (4) equipment operation constraint:
(a) Gas turbine operating constraints:
Figure GDA0003762154670000052
v in formula (7) gt，t Representing the natural gas inlet amount of the gas turbine in the t period; eta gt Representing the electrical efficiency of the gas turbine; j represents the natural gas heating value; h gt，t Representing the output thermal power of the gas turbine for a period t;
Figure GDA0003762154670000053
and
Figure GDA0003762154670000054
respectively representing the upper limit and the lower limit of the electric power of the gas turbine;
Figure GDA0003762154670000055
formula (8) wherein P gt，t And P gt，t-1 Respectively representing the power output values of the gas turbine at the time t and the time t-1;
Figure GDA0003762154670000056
and
Figure GDA0003762154670000057
representing minimum and maximum power variation values of the gas turbine during adjacent operational schedule periods, respectively;
(b) Gas boiler operation constraints
Figure GDA0003762154670000061
Eta in equation (9) gb Representing gas boiler efficiency;
Figure GDA0003762154670000062
and
Figure GDA0003762154670000063
respectively representing the upper limit and the lower limit of the power of the gas boiler;
Figure GDA0003762154670000064
h in the formula (10) gb，t And H gb，t-1 Respectively representing the power output values of the gas boiler at the time t and the time t-1;
Figure GDA0003762154670000065
and
Figure GDA0003762154670000066
respectively representing minimum and maximum power variation values of the gas boiler in adjacent operation scheduling periods;
(c) And (3) battery restraint:
Figure GDA0003762154670000067
SOC in equation (11) t And SOC t-1 Respectively representing the electric energy stored by the storage battery at the time t and the time t-1; Δ t is the time interval; r represents the battery itselfAn energy loss coefficient; eta bt，char Expressed as the charging efficiency of the battery; eta bt，dis Represents the discharge efficiency of the storage battery;
Figure GDA0003762154670000068
and
Figure GDA0003762154670000069
respectively representing the minimum and maximum capacities of the storage battery;
Figure GDA00037621546700000610
respectively representing the minimum and maximum charging power of the storage battery;
Figure GDA00037621546700000611
and
Figure GDA00037621546700000612
respectively representing the minimum and maximum discharge power of the storage battery; n is a radical of char，t And N dis，t Is a group of 0-1 variables, which represents the charging or discharging state of the storage battery, and the product of 0 represents that the two processes can not be carried out simultaneously;
Figure GDA00037621546700000613
p in formula (12) char，t And P char，t-1 Respectively representing the charging power of the storage battery at the time t and the time t-1;
Figure GDA00037621546700000614
and
Figure GDA00037621546700000615
respectively representing the minimum and maximum charging power change values of the storage battery in the adjacent operation scheduling period;
Figure GDA0003762154670000071
p in formula (13) dis，t And P dis，t-1 Respectively representing the discharge power of the storage battery at the time t and the time t-1;
Figure GDA0003762154670000072
and
Figure GDA0003762154670000073
respectively representing the minimum and maximum discharge power change values of the storage battery in the adjacent operation scheduling time period;
(d) And (4) heat storage tank restraint:
Figure GDA0003762154670000074
h in formula (14) tst，t And H tst，t-1 Respectively representing the heat stored in the heat storage tank at the time t and the time t-1; h tst，char And H tst，dis Representing the heat storage and release power of the heat storage tank; sigma is the energy loss rate of the heat storage tank; eta tst，char Indicating the heat storage efficiency; eta tst，dis Indicating the efficiency of heat release;
Figure GDA0003762154670000075
Figure GDA0003762154670000076
respectively representing the minimum and maximum heat release power of the heat storage tank;
Figure GDA0003762154670000077
respectively representing the minimum heat storage power and the maximum heat storage power of the heat storage tank;
Figure GDA0003762154670000078
and
Figure GDA0003762154670000079
is a group of variables from 0 to 1, which represents the heat storage or release state of the heat storage tank, and the product of 0 represents that the two processes can not be carried out simultaneously;
Figure GDA00037621546700000710
and
Figure GDA00037621546700000711
respectively representing the minimum and maximum capacities of the heat storage tank;
Figure GDA00037621546700000712
h in the formula (15) char，t And H char，t-1 Respectively representing the heat storage power of the heat storage tank at the time t and the time t-1;
Figure GDA00037621546700000713
and
Figure GDA00037621546700000714
respectively representing the minimum and maximum heat storage power change values of the heat storage tank in the adjacent operation scheduling periods;
Figure GDA00037621546700000715
h in formula (16) dis，t And H dis，t-1 Respectively representing the heat release power of the heat storage tank at the time t and the time t-1;
Figure GDA0003762154670000081
and
Figure GDA0003762154670000082
respectively representing the minimum and maximum heat release power change values of the heat storage tank in the adjacent operation scheduling period;
(e) Comprehensive demand response of the micro energy network:
Figure GDA0003762154670000083
p in formula (17) load，t′ And P load，t Respectively representing participation of micro energy networkMeeting the power load demand before and after the demand response;
Figure GDA0003762154670000084
and
Figure GDA0003762154670000085
respectively representing the electricity price at the time t' and the time t;
Figure GDA0003762154670000086
and
Figure GDA0003762154670000087
respectively representing the natural gas prices before and after the participation in the comprehensive demand response; e t (e) Representing the demand elasticity of the user t period on the electric power; e t (e, g) represents the cross-resilience of the power demand and the gas demand by the user during time t.
Preferably, the step S3 specifically includes:
the Markov decision process is represented by a quadruple (S, A, R, π);
wherein S is a state space, S t E S represents the state of the micro energy network control management system after the micro energy network control management system interacts with the environment in the period t:
Figure GDA0003762154670000088
a is an action space, a t E A represents the action which can be executed by the micro energy network control management system in the t period:
a t ＝(P ess，t ，H ess，t ，P grid，t ，P gt，t ，H gb，t ) (19)
p in formula (19) ess，t And H ess，t The charge and discharge/thermal power of the storage battery and the thermal storage tank are respectively expressed, and the charge and discharge actions of the storage battery are divided into K discrete charge and discharge selections according to the charge and discharge power range of the storage battery, namely:
Figure GDA0003762154670000091
in the formula (20)
Figure GDA0003762154670000092
Represents the K-th charge and discharge selection in the discrete motion space,
Figure GDA0003762154670000093
Figure GDA0003762154670000094
arranged in ascending order, first
Figure GDA0003762154670000095
Value of maximum discharge power
Figure GDA0003762154670000096
Last bit
Figure GDA0003762154670000097
Value of maximum charging power
Figure GDA0003762154670000098
Equally dividing the heat storage tank into K discrete heat charging and discharging selections according to the heat charging and discharging power range of the heat storage tank, namely:
Figure GDA0003762154670000099
in the formula (21)
Figure GDA00037621546700000910
Represents the K-th heat charge and discharge selection in the discrete motion space,
Figure GDA00037621546700000911
arranged in ascending order, first
Figure GDA00037621546700000912
Value of maximum heat release power
Figure GDA00037621546700000913
Last bit
Figure GDA00037621546700000914
Maximum heat storage power
Figure GDA00037621546700000917
R is a reward function consisting of t ∈R(s t ，a t ) Is represented by the formula (I) in which r t Representing the micro energy network in state s t Execute action a at once t Instant prize earned:
r t (s t ，a t )＝-(C+D) (22)
wherein C is the objective function and represents the operation cost of the micro energy network;
d represents the penalty cost when the power imbalance occurs in the operation process of the micro energy network or the overcharge or the overdischarge occurs to the energy storage equipment:
Figure GDA00037621546700000916
in the formula (23), c e And c h Respectively representing unit difference punishment cost of power supply and demand unbalance and heat energy supply and demand unbalance in the micro energy grid system; p is s，t And H s，t Respectively representing the total supply of electric power and heat energy in the micro energy grid during the t period; epsilon and theta respectively represent reasonable operation capacities of the storage battery and the heat storage tank; c. C bat And c tst Respectively representing unit punishment when the storage battery and the heat storage tank are overcharged, discharged and heated;
and pi is a strategy set and represents the mapping from the state space S of the micro energy network to the action space A.
Preferably, said state s t When determining, the micro energy network adopts the action value function Q for optimizing the quality of the scheduling action π (s, a) to evaluate, in particular:
Figure GDA0003762154670000101
e in the formula (24) π Expressing the expectation of timely return obtained by taking action under the guidance of a strategy pi; r is an instant reward; gamma is a discount factor representing the contribution of the decaying future reward to the current state value, and takes a value between 0 and 1.
Preferably, a cosine annealing algorithm is adopted to optimize the learning rate in the process of constructing the deep Q network model in the step S3.
Preferably, an empirical playback mechanism is adopted in the construction process of the deep Q network model, and/or a network parameter freezing mechanism is adopted in the construction process of the deep Q network model.
An industrial microgrid load optimization scheduling system with cogeneration, comprising:
the preprocessing module is used for preprocessing state parameters of all components of the industrial microgrid with combined heat and power generation;
the building module is used for building a micro energy network load optimization scheduling model based on the preprocessed state parameters;
the conversion module is used for converting the micro energy network load optimization scheduling model into a Markov decision process;
and the solving module is used for solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy containing combined heat and power generation.
(III) advantageous effects
The invention provides an industrial micro-grid load optimization scheduling method and system with combined heat and power generation. Compared with the prior art, the method has the following beneficial effects:
preprocessing state parameters of each component of the industrial microgrid; constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters; converting the micro energy network load optimization scheduling model into a Markov decision process; and solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy. The load optimization scheduling system of the industrial microgrid comprising the cogeneration equipment is constructed substantially, the influence of comprehensive demand response under the background of the energy Internet in the optimized scheduling process of the microgrid is considered, an optimized scheduling model is constructed by taking the minimization of the running cost of the microgrid as an optimization target from the actual production angle of an industrial enterprise, the energy consumption is saved for the operation of the enterprise, and the production cost is saved; the coupling operation characteristics of three energy flows of electricity, heat and gas are considered under the background of combined heat and power generation application, comprehensive demand response is taken as a link, the optimization of an energy utilization structure of a user is guided, and the demand response potential of the user side is excavated; the benign interaction of the multi-energy upper and lower level networks is promoted; the data-driven deep reinforcement learning method can effectively improve the efficiency and accuracy of optimized scheduling.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a block diagram illustrating a structure of an embodiment of an industrial enterprise microgrid with cogeneration equipment in the prior art;
fig. 2 is a schematic flowchart of a load optimization scheduling method for an industrial microgrid according to an embodiment of the present invention;
fig. 3 is a structural block diagram of an industrial microgrid load optimization scheduling system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application provides an industrial micro-grid load optimization scheduling method and system containing cogeneration, solves the technical problem that the existing micro-energy grid load optimization scheduling model cannot achieve real-time optimization scheduling, realizes the coupling operation of three energy flows of electricity, heat and gas, realizes the benign interaction of a multi-energy upper-level network and a multi-energy lower-level network, and can effectively improve the beneficial effects of optimization scheduling efficiency and accuracy by using a data-driven deep reinforcement learning method.
In order to solve the technical problems, the general idea of the embodiment of the application is as follows:
the method comprises the steps of preprocessing state parameters of all components of the industrial microgrid; constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters; converting the micro energy network load optimization scheduling model into a Markov decision process; and solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy. The industrial micro-grid load optimization scheduling system comprising the cogeneration equipment is constructed substantially, the influence of comprehensive demand response under the background of energy Internet in the micro-grid optimization scheduling process is considered, an optimization scheduling model is constructed by taking the minimization of the micro-grid operation cost as an optimization target from the practical production angle of industrial enterprises, the energy consumption is saved for enterprise operation, and the production cost is saved; the coupling operation characteristics of three energy flows of electricity, heat and gas are considered under the background of combined heat and power generation application, comprehensive demand response is taken as a link, the optimization of an energy utilization structure of a user is guided, and the demand response potential of the user side is excavated; the benign interaction of the multi-energy upper and lower level networks is promoted; the data-driven deep reinforcement learning method can effectively improve the efficiency and accuracy of optimized scheduling.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
As shown in fig. 2, an embodiment of the present invention provides an industrial microgrid load optimization scheduling method, including:
s1, preprocessing state parameters of each component of the industrial micro-grid;
s2, constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters;
s3, converting the micro energy network load optimization scheduling model into a Markov decision process;
and S4, solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy containing combined heat and power generation.
The embodiment of the invention essentially constructs an industrial micro-grid load optimization scheduling system comprising cogeneration equipment, considers the influence of comprehensive demand response under the background of energy Internet in the micro-grid optimization scheduling process, constructs an optimization scheduling model by taking the minimization of the micro-grid operation cost as an optimization target from the actual production angle of industrial enterprises, saves energy consumption for enterprise operation and saves production cost; the coupling operation characteristics of three energy flows of electricity, heat and gas are considered under the background of combined heat and power generation application, comprehensive demand response is taken as a link, the optimization of an energy utilization structure of a user is guided, and the demand response potential of the user side is excavated; the benign interaction of the multi-energy upper and lower level networks is promoted; the data-driven deep reinforcement learning method can effectively improve the efficiency and accuracy of optimized scheduling.
The embodiment is as follows:
in a first aspect, as shown in fig. 2, the present invention provides an industrial microgrid load optimization scheduling method, which specifically includes:
s1, preprocessing state parameters of each component of the industrial microgrid.
The state parameters at least comprise charging/discharging power, storage/discharging power, price, total heat load demand of a certain period of time in micro energy network load, total electric load demand of a certain period of time and the like of each component of the industrial micro grid load, and specific contents are introduced in the subsequent steps.
The preprocessing comprises data cleaning, data integration, data reduction, data transformation and the like.
And S2, constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters.
The method specifically comprises the following steps:
the construction process of the micro energy network load optimization scheduling model is based on a combined heat and power generation comprehensive energy demand response mechanism, and specifically comprises an objective function:
min cost C＝C 1 +C 2 +C 3 +C 4 (1)
wherein, C 1 Representing the electricity purchase and sale cost:
Figure GDA0003762154670000141
in the formula (2)
Figure GDA0003762154670000142
And
Figure GDA0003762154670000143
respectively representing the prices of electricity purchase and electricity sale of the micro energy grid from the large power grid; p is grid，t The electric quantity purchased/sold by the micro energy grid in the t period is represented, a positive value represents that the micro energy grid purchases electricity, and a negative value represents that the micro energy grid sells electricity to the large power grid; t represents the whole scheduling period;
C 2 represents the gas purchase cost:
Figure GDA0003762154670000144
g in formula (3) grid，t The natural gas quantity purchased from the natural gas network by the micro energy network in the t period is represented;
Figure GDA0003762154670000151
representing the natural gas price for the t period;
C 3 represents the depreciation cost of the energy storage device:
Figure GDA0003762154670000152
the first term in the formula (4) is the electric power storageDepreciation cost of the pool, wherein P char，t Represents the battery charging power for a period of t; p dis，t Represents the battery discharge power in the t period; because the depreciation cost of the energy storage battery is in direct proportion to the energy storage charging and discharging electric quantity, the ratio of the depreciation cost to the energy storage charging and discharging electric quantity is called depreciation coefficient and is recorded as k, and the calculation mode is
Figure GDA0003762154670000153
Figure GDA0003762154670000154
Price in the formula ess The price of the energy storage battery is shown,
Figure GDA0003762154670000155
the rated capacity of the energy storage battery is represented, and the cycle life of the battery is represented by L;
the second term is the depreciation cost of the heat storage tank, wherein h is the depreciation coefficient of the heat storage tank, and the calculation mode is
Figure GDA0003762154670000156
price tst Which represents the price of the heat storage tank,
Figure GDA0003762154670000157
the rated capacity of the heat storage tank is shown, and M represents the cycle life of the heat storage tank;
C 4 represents the equipment operation and maintenance cost:
C 4 ＝P gt，t *K gt +(|P char，t |+|P dis，t |)*K bt +P pv，t *K pv +H gb，t *K gb +H hr，t *K hr (5)
formula (5) wherein P gt，t Electrical power representing a period t of the gas turbine; k bt Represents a gas turbine operating maintenance cost; (| P) char，t |+|P dis，t |) represents the charge-discharge power of the storage battery at t period; k bt Representing the running and maintenance cost of the storage battery; p pv，t Output electric power representing a period t of the photovoltaic panel; k pv Representing the operating and maintenance cost of the photovoltaic panel; h gb，t Indicating period t of the gas boilerThe output power of (a); k gb The operating and maintenance costs for the gas boiler; h hr，t Output power, K, representing a period t of the waste heat recovery device hr Representing the operating and maintenance costs of the waste heat recovery device.
The construction of the micro energy network load optimization scheduling model further comprises the following constraint conditions:
A. power supply and demand balance constraint:
Figure GDA0003762154670000158
p in formula (6) grid，t Represents the transmission power between the large power grid and the micro energy grid in the period of t, P pv，t Output electric power, P, representing the t period of the photovoltaic panel ess，t Representing the charge-discharge power, P, of the accumulator during a period t ess，t ＝P dis，t -P char，t ，P gt，t Electric power, P, representing the t period of the gas turbine load，t Representing the total electrical load demand in the micro energy grid during time t;
H hr，t the output thermal power of the waste heat recovery device in a t period is represented; h gb，t Representing the output thermal power of the gas boiler for a period t; h dis，t Represents the heat release power of the heat storage tank t period; h char，t Represents the heat storage power of the heat storage tank t period; h load，t Representing the total heat load demand in the micro energy network in the period t;
B. and (4) equipment operation constraint:
(a) Gas turbine operating constraints:
Figure GDA0003762154670000161
v in formula (7) gt，t Representing the natural gas inlet amount of the gas turbine in the t period; eta gt Representing the electrical efficiency of the gas turbine; j represents the natural gas heating value; h gt，t Representing the output thermal power of the gas turbine for a period t;
Figure GDA0003762154670000162
and
Figure GDA0003762154670000163
respectively representing the upper limit and the lower limit of the electric power of the gas turbine;
Figure GDA0003762154670000164
formula (8) wherein P gt，t And P gt，t-1 Respectively representing the power output values of the gas turbine at the time t and the time t-1;
Figure GDA0003762154670000165
and
Figure GDA0003762154670000166
representing minimum and maximum power variation values of the gas turbine during adjacent operational schedule periods, respectively;
(b) Gas boiler operation constraints
Figure GDA0003762154670000167
Eta in equation (9) gb Representing the gas boiler efficiency;
Figure GDA0003762154670000168
and
Figure GDA0003762154670000169
respectively representing the upper limit and the lower limit of the power of the gas boiler;
Figure GDA0003762154670000171
h in the formula (10) gb，t And H gb，t-1 Respectively representing the power output values of the gas boiler at the time t and the time t-1;
Figure GDA0003762154670000172
and
Figure GDA0003762154670000173
respectively representing minimum and maximum power variation values of the gas boiler in adjacent operation scheduling periods;
(c) And (3) battery restraint:
Figure GDA0003762154670000174
SOC in equation (11) t And SOC t-1 Respectively representing the electric energy stored by the storage battery at the time t and the time t-1; Δ t is the time interval; r represents the self energy loss coefficient of the storage battery; eta bt，char Expressed as the charging efficiency of the battery; eta bt，dis The discharge efficiency of the storage battery is represented;
Figure GDA0003762154670000175
and
Figure GDA0003762154670000176
respectively representing the minimum and maximum capacities of the storage battery;
Figure GDA0003762154670000177
respectively representing the minimum and maximum charging power of the storage battery;
Figure GDA0003762154670000178
and
Figure GDA0003762154670000179
respectively representing the minimum and maximum discharge power of the storage battery; n is a radical of char，t And N dis，t Is a group of 0-1 variables which represent the charging or discharging state of the storage battery, and the product of 0 represents that the two processes can not be carried out simultaneously;
Figure GDA00037621546700001710
p in formula (12) char，t And P char，t-1 Respectively representing the charging power of the storage battery at the time t and the time t-1;
Figure GDA00037621546700001715
and
Figure GDA00037621546700001711
respectively representing the minimum and maximum charging power change values of the storage battery in the adjacent operation scheduling period;
Figure GDA00037621546700001712
p in formula (13) dis，t And P dis，t-1 Respectively representing the discharge power of the storage battery at the time t and the time t-1;
Figure GDA00037621546700001713
and
Figure GDA00037621546700001714
respectively representing the minimum and maximum discharge power change values of the storage battery in the adjacent operation scheduling time period;
(d) And (3) heat storage tank restraint:
Figure GDA0003762154670000181
h in formula (14) tst，t And H tst，t-1 Respectively representing the heat stored in the heat storage tank at the time t and the time t-1; h tst，char And H tst，dis Representing the heat storage and release power of the heat storage tank; sigma is the energy loss rate of the heat storage tank; eta tst，char Indicating the heat storage efficiency; eta tst，dis Indicating the efficiency of heat release;
Figure GDA0003762154670000182
Figure GDA0003762154670000183
respectively representing minimum and minimum of heat storage tankLarge heat release power;
Figure GDA0003762154670000184
respectively representing the minimum heat storage power and the maximum heat storage power of the heat storage tank;
Figure GDA0003762154670000185
and
Figure GDA0003762154670000186
is a group of variables from 0 to 1, which represents the heat storage or release state of the heat storage tank, and the product of 0 represents that the two processes can not be carried out simultaneously;
Figure GDA0003762154670000187
and
Figure GDA0003762154670000188
respectively representing the minimum and maximum capacities of the thermal storage tank;
Figure GDA0003762154670000189
h in the formula (15) char，t And H char，t-1 Respectively representing the heat storage power of the heat storage tank at the time t and the time t-1;
Figure GDA00037621546700001810
and
Figure GDA00037621546700001811
respectively representing the minimum and maximum heat storage power change values of the heat storage tank in the adjacent operation scheduling time period;
Figure GDA00037621546700001812
h in formula (16) dis，t And H dis，t-1 Respectively representing the heat release power of the heat storage tank at the time t and the time t-1;
Figure GDA00037621546700001813
and
Figure GDA00037621546700001814
respectively representing the minimum and maximum heat release power change values of the heat storage tank in the adjacent operation scheduling period;
(e) Comprehensive demand response of the micro energy network:
Figure GDA0003762154670000191
p in formula (17) load，t′ And P load，t Respectively representing the power load demands before and after the micro energy network participates in the comprehensive demand response;
Figure GDA0003762154670000192
and
Figure GDA0003762154670000193
respectively representing the electricity price at the time t' and the time t;
Figure GDA0003762154670000194
and
Figure GDA0003762154670000195
respectively representing the natural gas prices before and after the natural gas prices participate in the comprehensive demand response; e t (e) Representing the demand elasticity of the user t period on the electric power; e t (e, g) represents the cross-over resilience of the customer t period power demand and gas demand.
And S3, converting the micro energy network load optimization scheduling model into a Markov decision process. The method specifically comprises the following steps:
and converting the constructed micro energy network load optimization scheduling model into a basic framework of a reinforcement learning algorithm, namely a Markov Decision Process (MDP).
The core of the industrial enterprise micro energy network load optimization scheduling strategy based on reinforcement learning provided by the embodiment of the invention is a sequential decision problem, and the attention is paid to the selection of starting and stopping and charging and discharging actions of each energy supply and energy storage device in each decision stage in the industrial enterprise micro energy network load and how to optimize the task sequence of the whole system through the selection.
The problems that the efficiency is low, the model universality is poor, the load and other data need to be predicted before scheduling and the like in the existing technical scheme of the micro energy network load optimization scheduling are solved. Therefore, the scheme of the embodiment of the invention solves the sequential decision process through deep reinforcement learning, so that the micro energy network load optimization scheduling model needs to be converted into a Markov decision process.
The Markov decision process is represented by a quadruple (S, A, R, π). When the microgrid optimization scheduling model is converted into a markov decision process, the objective function and the constraint condition contained in the constructed model need to be correspondingly integrated into an MDP reward function, a state space and an action space, wherein the meaning of each element and the corresponding content in the invention are as follows:
s is a state space, S t E S represents the state of the micro energy network control management system after the micro energy network control management system interacts with the environment in the period t:
Figure GDA0003762154670000201
a is an action space, a t And E A represents the action which can be executed by the micro energy network load management system in the t period:
a t ＝(P ess，t ，H ess，t ，P grid，t ，P gt，t ，H gb，t ) (19)
p in formula (19) ess，t And H ess，t The charge and discharge/thermal power of the storage battery and the thermal storage tank are respectively expressed, and the charge and discharge actions of the storage battery are divided into K discrete charge and discharge selections according to the charge and discharge power range of the storage battery, namely:
Figure GDA0003762154670000202
in the formula (20)
Figure GDA0003762154670000203
Represents the K-th charge and discharge selection in the discrete motion space,
Figure GDA0003762154670000204
Figure GDA0003762154670000205
arranged in ascending order, first
Figure GDA0003762154670000206
Value of maximum discharge power
Figure GDA0003762154670000207
Last bit
Figure GDA0003762154670000208
Value of maximum charging power
Figure GDA0003762154670000209
Equally dividing the heat storage tank into K discrete heat charging and discharging selections according to the heat charging and discharging power range of the heat storage tank, namely:
Figure GDA00037621546700002010
in the formula (21)
Figure GDA00037621546700002011
Represents the K-th heat charge and discharge selection in the discrete motion space,
Figure GDA0003762154670000211
arranged in ascending order, first
Figure GDA0003762154670000212
Value of maximum heat release power
Figure GDA0003762154670000213
Last bit
Figure GDA0003762154670000214
Maximum heat storage power
Figure GDA0003762154670000215
R is a reward function consisting of rt ∈R(s t ，a t ) Is shown in which rt Representing the micro energy network in state s t Execute action a at once t Instant prize earned:
r t (s t ，a t )＝-(C+D) (22)
wherein C is the objective function and represents the operation cost of the micro energy network; d represents the penalty cost when the power imbalance occurs in the operation process of the micro energy network or the overcharge or the overdischarge occurs to the energy storage equipment:
Figure GDA0003762154670000216
in the formula (23), c e And c h Respectively representing unit difference punishment cost of power supply and demand unbalance and heat energy supply and demand unbalance in the micro energy grid system; p s，t And H s，t Respectively representing the total supply of electric power and heat energy in the micro energy grid during the t period; epsilon and theta respectively represent reasonable operation capacities of the storage battery and the heat storage tank; c. C bat And c tst Respectively representing unit punishments when the storage battery and the heat storage tank are overcharged, discharged and heated;
and pi is a strategy set and represents the mapping from the state space S of the micro energy network to the action space A.
As described above, embodiments of the present invention aim to minimize the system cost of micro energy grid load during optimal scheduling. When the state s t When the micro energy network load is determined, the micro energy network load adopts the action value function Q for optimizing the scheduling action π (s, a) to evaluate, in particular:
Figure GDA0003762154670000217
e in the formula (24) π Expressing the expectation of timely return obtained by taking action under the guidance of a strategy pi; r is an instant reward; gamma is a discount factor representing the contribution of decaying future returns to the current state value, and is generally between 0 and 1.
And S4, solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy containing combined heat and power generation.
According to the current state s of the system in each scheduling period t Selecting a scheduling action a by adopting a pre-trained Deep Q Network (DQN) t And executing the action, and then the system enters the next state s t+1 And obtain an instant prize r t 。
And continuously and repeatedly executing the actions, namely obtaining the system state information at the time of t +1 as a new sample, and entering the next decision stage, so that a complete micro energy source network load optimization scheduling strategy can be obtained.
The micro-energy network load optimization scheduling mentioned in the embodiment of the invention is a complex process of multi-equipment linkage and multi-energy coupling operation, is essentially a sequential decision problem, and is suitable for solving by adopting a reinforcement learning method.
When the problem of optimal scheduling of micro-energy grid loads is solved by reinforcement learning, the photovoltaic power generation power and load requirements do not need to be predicted in advance like the traditional micro-grid day-ahead scheduling method, the uncertainty influence in the prediction process is reduced, and the method has the unique advantage of not depending on the source load uncertainty distribution knowledge in the micro-energy grid loads.
In particular, in the embodiment of the invention, the state space of the optimal scheduling problem of cogeneration of industrial enterprises comprises continuous variables, and the traditional reinforcement learning method is often poor in effect due to dimension disaster problem during processing.
In contrast, the embodiment of the invention provides a method for performing energy management and optimization strategy selection on an industrial enterprise cogeneration system by using a deep Q learning algorithm in deep reinforcement learning, the performance of the algorithm is improved by using an experience playback mechanism and a freezing network parameter mechanism, and an action value function Q is approximately expressed by using a deep neural network, so that the dimension disaster of the traditional reinforcement learning method is solved, the real-time energy management and optimization of the industrial enterprise cogeneration micro-energy network load are realized, the problems of difficult modeling, slow convergence of the traditional algorithm operation and the like of the operation of the industrial enterprise cogeneration micro-energy network load with random and intermittent renewable energy, energy storage equipment and comprehensive demand response are effectively solved, and the real-time optimization is realized.
The input of DQN is state S, the state space vector has several input nodes in the dimension, the output is each action a in the action space i The corresponding Q value Q (s, a; omega), the output node number is equal to the action total number. The deep Q learning algorithm approximates the action value function by using a deep Q network with a network parameter omega on the basis of the traditional Q learning algorithm, and can solve the problem of dimension disaster when the traditional Q learning algorithm processes continuous variables in a state space.
In the neural network training process, a learning rate cosine annealing (learning rate cosine annealing) technology is adopted to optimize the learning rate in the training process. When the loss function is closer to the global minimum value in the training process, the cosine annealing technology can reduce the learning rate through the cosine function, so that the situations that the loss value of the model is vibrated and the convergence is slowed do not occur.
The sample data is generated by interaction between the micro energy network load optimization scheduling control system and the environment in the training process, and the samples have relevance. In order to reduce sample correlation in the training process and improve training stability, an experience playback mechanism and a parameter freezing mechanism are introduced into a deep Q learning algorithm, and a cosine annealing technology is used in the neural network parameter adjusting process.
The experience playback mechanism is that each MDP tuple (S, A, R, pi) generated in the training process is stored in an experience playback pool, then data is extracted from the pool by using a uniform random sampling method, and the extracted data is used for training a neural network. The empirical playback mechanism can effectively break the correlation between the data, and the sampling can increase the use efficiency of the data, so that the training process is more robust.
The parameter freezing mechanism is characterized in that two deep neural networks with completely consistent structures, namely a current value network and a target value network, are introduced, parameters of the current value network are updated in real time in training, and parameters of the target value network are directly copied from the current value network at fixed step numbers. The training process can be stabilized by freezing the target value network for a period of time and then directly copying the parameters from the current value network.
Before the neural network is applied to the real-time optimal scheduling of the load of the micro energy network, firstly, historical operation data of the load of the micro energy network is calculated(s) t ，a t ，r，s t+1 ) The form of (2) is stored in an experience pool to form a playback memory sequence. During training, small batches of experience samples are randomly extracted from the sequence every time, and network parameters are updated by adopting an Adaptive motion Estimation (Adam) algorithm. After the training is finished, the DQN algorithm parameters obtained by training are fixed.
According to the embodiment of the invention, a deep Q network method based on data driving is adopted to solve the microgrid optimal scheduling. The Q function is approximately expressed by utilizing the neural network, the optimization scene of the complex state transfer process in the industrial microgrid comprising the cogeneration equipment can be efficiently processed, and the optimization of three energy flow coupling dynamic processes of electricity, gas and heat with slow change characteristics can be well depicted. On the other hand, in the solving process of the deep Q network based on data driving, compared with the traditional algorithm, the deep Q network based on data driving has the advantages of being faster in convergence speed, easier to obtain the global optimal solution and more accurate in optimized scheduling result.
In a second aspect, as shown in fig. 3, an embodiment of the present invention further provides an industrial microgrid load optimization scheduling system, including:
the preprocessing module is used for preprocessing the state parameters of each component of the industrial microgrid;
the building module is used for building a micro energy network load optimization scheduling model based on the preprocessed state parameters;
the conversion module is used for converting the micro energy network load optimization scheduling model into a Markov decision process;
and the solving module is used for solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy containing combined heat and power generation.
It can be understood that, the industrial microgrid load optimization scheduling system provided by the invention corresponds to the industrial microgrid load optimization scheduling method provided by the invention, and the explanation, example, beneficial effects and other parts of relevant contents can refer to the corresponding parts in the industrial microgrid load optimization scheduling method, and are not described herein again.
In summary, compared with the prior art, the method has the following beneficial effects:
1. the method comprises the steps of preprocessing state parameters of all components of the industrial microgrid; constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters; converting the micro energy network load optimization scheduling model into a Markov decision process; and solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy. The industrial micro-grid load optimization scheduling system comprising the cogeneration equipment is constructed substantially, the influence of comprehensive demand response under the background of energy Internet in the micro-grid optimization scheduling process is considered, an optimization scheduling model is constructed by taking the minimization of the micro-grid operation cost as an optimization target from the practical production angle of industrial enterprises, the energy consumption is saved for enterprise operation, and the production cost is saved; the coupling operation characteristics of three energy flows of electricity, heat and gas are considered in the context of combined heat and power generation application, comprehensive demand response is taken as a link, energy utilization structure optimization of a user is guided, and demand response potential of the user side is excavated; the benign interaction of the multi-energy upper and lower level networks is promoted; the data-driven deep reinforcement learning method can effectively improve the efficiency and accuracy of optimized scheduling.
2. In the embodiment of the invention, in the neural network training process, a learning rate cosine annealing (learning rate cosine annealing) technology is adopted to optimize the learning rate in the training process. When the loss function is closer to the global minimum value in the training process, the cosine annealing technology can reduce the learning rate through the cosine function, so that the situations that the loss value oscillates and the convergence becomes slow can not occur in the model.
3. The method and the device solve the microgrid optimal scheduling by adopting a data-driven deep Q network method. The Q function is approximately expressed by utilizing the neural network, the optimization scene of the complex state transfer process in the industrial microgrid comprising the cogeneration equipment can be efficiently processed, and the optimization of three energy flow coupling dynamic processes of electricity, gas and heat with slow change characteristics can be well depicted. On the other hand, in the solving process of the deep Q network based on data driving, compared with the traditional algorithm, the deep Q network based on data driving has the advantages of being faster in convergence speed, easier to obtain the global optimal solution and more accurate in optimized scheduling result.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (7)

1. The method for optimizing and scheduling the load of the industrial microgrid with cogeneration is characterized by comprising the following steps of:
s1, preprocessing state parameters of all components of an industrial micro-grid containing cogeneration;
s2, constructing a micro energy network load optimization scheduling model based on the preprocessed state parameters;
s3, converting the micro energy network load optimization scheduling model into a Markov decision process;
s4, solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro-energy grid load optimization scheduling strategy containing combined heat and power generation;
the building process of the micro energy network load optimization scheduling model in the step S2 is based on a combined heat and power generation comprehensive energy demand response mechanism, and specifically includes an objective function:
min cost C＝C 1 +C 2 +C 3 +C 4 (1)
wherein, C 1 Representing the electricity purchase and sale cost:
Figure FDA0003762154660000011
in the formula (2)
Figure FDA0003762154660000012
And
Figure FDA0003762154660000013
respectively representing the prices of electricity purchase and electricity sale of the micro energy grid from the large power grid; p grid,t The electricity purchasing/selling quantity of the micro energy network in the t period is represented, a positive value represents that the micro energy network purchases electricity, and a negative value represents that the micro energy network sells electricity to the large power grid; t represents the whole scheduling period;
C 2 represents the gas purchase cost:
Figure FDA0003762154660000014
g in the formula (3) grid,t Representing the amount of natural gas purchased from a natural gas network by the micro energy network in the t period;
Figure FDA0003762154660000015
representing the natural gas price for the t period;
C 3 represents the depreciation cost of the energy storage device:
Figure FDA0003762154660000021
the first term in equation (4) is the cost of battery depreciation, where P car,t Represents the battery charging power for a period t; p dis,t Represents the battery discharge power in the t period; because the depreciation cost of the energy storage battery is in direct proportion to the energy storage charging and discharging electric quantity, the ratio of the depreciation cost to the energy storage charging and discharging electric quantity is called depreciation coefficient and is recorded as k, and the calculation mode is
Figure FDA0003762154660000022
Figure FDA0003762154660000023
Price in the formula ess The price of the energy storage battery is shown,
Figure FDA0003762154660000024
the rated capacity of the energy storage battery is shown, and the cycle life of the battery is shown by L;
the second term is the depreciation cost of the heat storage tank, wherein h is the depreciation coefficient of the heat storage tank, and the calculation mode is
Figure FDA0003762154660000025
price tst Which represents the price of the heat storage tank,
Figure FDA0003762154660000026
the rated capacity of the heat storage tank is shown, and M represents the cycle life of the heat storage tank;
C 4 represents the equipment operation and maintenance cost:
C 4 ＝P gt,t *K gt +(|P char,t |+|P dis,t |)*K bt +P pv,t *K pv +H gb,t *K gb +H hr,t *K hr (5)
formula (5) wherein P gt,t Electrical power representing a gas turbine time period t; k bt Represents a gas turbine operating maintenance cost; (| P) char,t |+|P dis,t |) represents the charge-discharge power of the storage battery at t period; k bt Representing the running and maintenance cost of the storage battery; p pv,t Output electric power representing a period t of the photovoltaic panel; k pv Representing the operation and maintenance cost of the photovoltaic panel; h gb,t Output power representing a period t of the gas boiler; k gb The operating and maintenance costs of the gas boiler; h hr,t Output power, K, representing a period t of the waste heat recovery device hr Representing the operating and maintenance costs of the waste heat recovery device.
2. The load optimization scheduling method for the industrial microgrid with cogeneration of heat and power as claimed in claim 1, wherein the building process of the load optimization scheduling model for the micro energy grid in the step S2 further comprises constraint conditions:
A. power supply and demand balance constraint:
Figure FDA0003762154660000031
p in formula (6) grid,t Represents the transmission power between the large power grid and the micro energy grid in the period of t, P pv,t Output electric power, P, representing the t period of the photovoltaic panel ess,t Representing the charge-discharge power, P, of the accumulator during a period t ess,t ＝P dis,t -P char,t ，P gt,t Electric power, P, representing the t period of the gas turbine load,t Representing total electrical load demand for time period t in a micro energy grid；
H hr,t The output thermal power of the waste heat recovery device in a t period is represented; h gb,t Representing the output thermal power of the gas boiler during the t period; h dis,t Represents the heat release power of the heat storage tank t period; h char,t Represents the heat storage power of the heat storage tank t period; h load,t Representing the total heat load demand in the micro energy network in the time period t;
B. and (4) equipment operation constraint:
(a) Gas turbine operating constraints:
Figure FDA0003762154660000032
v in formula (7) gt,t Representing the natural gas inlet amount of the gas turbine in the t period; eta gt Representing the electrical efficiency of the gas turbine; j represents the natural gas heating value; h gt,t Representing the output thermal power of the gas turbine for a period t;
Figure FDA0003762154660000033
and
Figure FDA0003762154660000034
respectively representing the upper limit and the lower limit of the electric power of the gas turbine;
Figure FDA0003762154660000035
formula (8) wherein P gt,t And P gt,t-1 Respectively representing the power output values of the gas turbine at the time t and the time t-1;
Figure FDA0003762154660000036
and
Figure FDA0003762154660000037
representing minimum and maximum power variation values of the gas turbine during adjacent operational schedule periods, respectively;
(b) Gas boiler operation constraints
Figure FDA0003762154660000041
Eta in equation (9) gb Representing the gas boiler efficiency;
Figure FDA0003762154660000042
and
Figure FDA0003762154660000043
respectively representing the upper limit and the lower limit of the power of the gas boiler;
Figure FDA0003762154660000044
h in the formula (10) gb,t And H gb,t-1 Respectively representing the power output values of the gas boiler at the time t and the time t-1;
Figure FDA0003762154660000045
and
Figure FDA0003762154660000046
respectively representing minimum and maximum power variation values of the gas boiler in adjacent operation scheduling periods;
(c) And (3) battery restraint:
Figure FDA0003762154660000047
SOC in equation (11) t And SOC t-1 Respectively representing the electric energy stored by the storage battery at the time t and the time t-1; Δ t is the time interval; r represents the self energy loss coefficient of the storage battery; eta bt,char Expressed as the charging efficiency of the battery; eta bt,dis Represents the discharge efficiency of the storage battery;
Figure FDA0003762154660000048
and
Figure FDA0003762154660000049
respectively representing the minimum and maximum capacities of the storage battery;
Figure FDA00037621546600000410
respectively representing the minimum and maximum charging power of the storage battery;
Figure FDA00037621546600000411
and
Figure FDA00037621546600000412
respectively representing the minimum and maximum discharge power of the storage battery; n is a radical of hydrogen char,t And N dis,t Is a group of 0-1 variables, which represents the charging or discharging state of the storage battery, and the product of 0 represents that the two processes can not be carried out simultaneously;
Figure FDA00037621546600000413
p in formula (12) char,t And P char,t-1 Respectively representing the charging power of the storage battery at the time t and the time t-1;
Figure FDA00037621546600000414
and
Figure FDA00037621546600000415
respectively representing the minimum and maximum charging power change values of the storage battery in the adjacent operation scheduling period;
Figure FDA0003762154660000051
p in formula (13) dis,t And P dis,t-1 Respectively representing the time of the storage battery at the t period and the t-1 momentThe discharge power of (d);
Figure FDA0003762154660000052
and
Figure FDA0003762154660000053
respectively representing the minimum and maximum discharge power change values of the storage battery in the adjacent operation scheduling periods;
(d) And (4) heat storage tank restraint:
Figure FDA0003762154660000054
h in formula (14) tst,t And H tst,t-1 Respectively representing the heat stored in the heat storage tank at the time t and the time t-1; h tst,char And H tst,dis Representing the heat storage and release power of the heat storage tank; sigma is the energy loss rate of the heat storage tank; eta tst,char Indicating the heat storage efficiency; eta tst,dis Indicating the efficiency of heat release;
Figure FDA0003762154660000055
Figure FDA0003762154660000056
respectively representing the minimum and maximum heat release power of the heat storage tank;
Figure FDA0003762154660000057
respectively representing the minimum heat storage power and the maximum heat storage power of the heat storage tank;
Figure FDA0003762154660000058
and
Figure FDA0003762154660000059
is a group of variables from 0 to 1, which represents the heat storage or release state of the heat storage tank, and the product of 0 represents that the two processes can not be carried out simultaneously;
Figure FDA00037621546600000510
and
Figure FDA00037621546600000511
respectively representing the minimum and maximum capacities of the heat storage tank;
Figure FDA00037621546600000512
h in the formula (15) char,t And H char,t-1 Respectively representing the heat storage power of the heat storage tank at the time t and the time t-1;
Figure FDA00037621546600000513
and
Figure FDA00037621546600000514
respectively representing the minimum and maximum heat storage power change values of the heat storage tank in the adjacent operation scheduling time period;
Figure FDA00037621546600000515
h in the formula (16) dis,t And H dis,t-1 Respectively representing the heat release power of the heat storage tank at the time t and the time t-1;
Figure FDA0003762154660000061
and
Figure FDA0003762154660000062
respectively representing the minimum and maximum heat release power change values of the heat storage tank in the adjacent operation scheduling period;
(e) Comprehensive demand response of the micro energy network:
Figure FDA0003762154660000063
p in formula (17) load,t′ And P load,t Respectively representing the power load demands before and after the micro energy network participates in the comprehensive demand response;
Figure FDA0003762154660000064
and
Figure FDA0003762154660000065
respectively representing the electricity price at the time t' and the time t;
Figure FDA0003762154660000066
and
Figure FDA0003762154660000067
respectively representing the natural gas prices before and after the natural gas prices participate in the comprehensive demand response; e t (e) Representing the demand elasticity of the user t period on the electric power; e t (e, g) represents the cross-resilience of the power demand and the gas demand by the user during time t.
3. The load optimization scheduling method for the industrial microgrid with cogeneration as claimed in claim 2, wherein the step S3 specifically comprises:
the Markov decision process is represented by a quadruple (S, A, R, π);
wherein S is a state space, S t E S represents the state of the micro energy network control management system after the micro energy network control management system interacts with the environment in the period t:
Figure FDA0003762154660000068
a is an action space, a t E A represents the action which can be executed by the micro energy network control management system in the t period:
a t ＝(P ess,t ,H ess,t ,P grid,t ,P gt,t ,H gb,t ) (19)
p in formula (19) ess,t And H ess,t Respectively represent the stored electricityThe charging and discharging/thermal power of the battery and the heat storage tank divides the charging and discharging actions of the storage battery into K discrete charging and discharging choices according to the charging and discharging power range of the storage battery, namely:
Figure FDA0003762154660000071
in the formula (20)
Figure FDA0003762154660000072
Represents the K-th charge and discharge selection in the discrete motion space,
Figure FDA0003762154660000073
Figure FDA0003762154660000074
arranged in ascending order, first
Figure FDA0003762154660000075
Value of maximum discharge power
Figure FDA0003762154660000076
Last bit
Figure FDA0003762154660000077
Value of maximum charging power
Figure FDA0003762154660000078
Equally dividing the heat storage tank into K discrete heat charging and discharging selections according to the heat charging and discharging power range of the heat storage tank, namely:
Figure FDA0003762154660000079
in the formula (21)
Figure FDA00037621546600000710
Represents the K-th heat charge and discharge selection in the discrete motion space,
Figure FDA00037621546600000711
arranged in ascending order, first
Figure FDA00037621546600000712
Value of maximum heat release power
Figure FDA00037621546600000713
Last bit
Figure FDA00037621546600000714
Maximum heat storage power
Figure FDA00037621546600000715
R is a reward function consisting of t ∈R(s t ,a t ) Is represented by the formula (I) in which r t Representing the micro energy network in state s t While performing action a t Instant prize earned:
r t (s t ,a t )＝-(C+D) (22)
wherein C is the objective function and represents the operation cost of the micro energy network;
d represents the penalty cost when the power imbalance occurs in the operation process of the micro energy network or the overcharge or the overdischarge occurs to the energy storage equipment:
Figure FDA00037621546600000716
in the formula (23), c e And c h Respectively representing unit difference punishment cost of power supply and demand unbalance and heat energy supply and demand unbalance in the micro energy grid system; p is s,t And H s,t Respectively representing the total supply of electric power and heat energy in the micro energy grid during the t period; ε and θ represent storage battery and heat storage, respectivelyReasonable operating capacity of the tank; c. C bat And c tst Respectively representing unit punishment when the storage battery and the heat storage tank are overcharged, discharged and heated;
and pi is a strategy set and represents the mapping from the state space S of the micro energy network to the action space A.
4. The industrial microgrid load optimization scheduling method comprising cogeneration according to claim 3, characterized in that the state s t When the micro energy network determines that the micro energy network adopts the optimal scheduling action, the micro energy network adopts an action value function Q π (s, a) to evaluate, in particular:
Figure FDA0003762154660000081
e in the formula (24) π Expressing the expectation of timely return obtained by taking action under the guidance of a strategy pi; r is an instant reward; gamma is a discount factor representing the contribution of the decaying future reward to the current state value, and takes a value between 0 and 1.
5. The load optimization scheduling method for the industrial microgrid with cogeneration as recited in claim 1, wherein a cosine annealing algorithm is adopted to optimize a learning rate in the construction process of the deep Q network model in the step S3.
6. The load optimization scheduling method for the industrial microgrid with cogeneration of heat and power as claimed in any one of claims 1 to 5, wherein the construction process of the deep Q network model further comprises an empirical playback mechanism and/or a freezing network parameter mechanism.
7. An industrial microgrid load optimization scheduling system containing cogeneration is characterized by comprising:
the preprocessing module is used for preprocessing state parameters of all components of the industrial microgrid with combined heat and power generation;
the building module is used for building a micro energy network load optimization scheduling model based on the preprocessed state parameters;
the conversion module is used for converting the micro energy network load optimization scheduling model into a Markov decision process;
the solving module is used for solving the Markov decision process by adopting a pre-trained deep Q network model to obtain a micro energy network load optimization scheduling strategy containing combined heat and power generation;
the building process of the micro energy network load optimization scheduling model in the building module is based on a comprehensive energy demand response mechanism of cogeneration, and specifically comprises an objective function:
min cost C＝C 1 +C 2 +C 3 +C 4 (1)
wherein, C 1 Representing the electricity purchase and sale cost:
Figure FDA0003762154660000091
in the formula (2)
Figure FDA0003762154660000092
And
Figure FDA0003762154660000093
respectively representing the prices of electricity purchase and electricity sale of the micro energy grid from the large power grid; p grid,t The electricity purchasing/selling quantity of the micro energy network in the t period is represented, a positive value represents that the micro energy network purchases electricity, and a negative value represents that the micro energy network sells electricity to the large power grid; t represents the whole scheduling period;
C 2 represents the gas purchase cost:
Figure FDA0003762154660000094
g in formula (3) grid,t Representing the amount of natural gas purchased from a natural gas network by the micro energy network in the t period;
Figure FDA0003762154660000095
representing the natural gas price for the t period;
C 3 represents the depreciation cost of the energy storage device:
Figure FDA0003762154660000096
the first term in equation (4) is the cost of battery depreciation, where P char,t Represents the battery charging power for a period t; p dis,t Represents the battery discharge power in the t period; because the depreciation cost of the energy storage battery is in direct proportion to the energy storage charging and discharging electric quantity, the ratio of the depreciation cost to the energy storage charging and discharging electric quantity is called depreciation coefficient and is recorded as k, and the calculation mode is
Figure FDA0003762154660000101
Figure FDA0003762154660000102
Price in the formula ess The price of the energy storage battery is shown,
Figure FDA0003762154660000103
the rated capacity of the energy storage battery is shown, and the cycle life of the battery is shown by L;
the second term is the depreciation cost of the heat storage tank, wherein h is the depreciation coefficient of the heat storage tank, and the calculation mode is
Figure FDA0003762154660000104
price tst Which represents the price of the heat storage tank,
Figure FDA0003762154660000105
the rated capacity of the heat storage tank is shown, and M represents the cycle life of the heat storage tank;
C 4 represents the equipment operation and maintenance cost:
C 4 ＝P gt,t *K gt +(|P char,t |+|P dis,t |)*K bt +P pv,t *K pv +H gb,t *K gb +H hr,t *K hr (5)
formula (5) wherein P gt,t Electrical power representing a gas turbine time period t; k bt Represents a gas turbine operating maintenance cost; (| P) char,t |+|P dis,t |) represents the charge-discharge power of the storage battery at t period; k bt Representing the running and maintenance cost of the storage battery; p pv,t Output electric power representing a period t of the photovoltaic panel; k pv Representing the operating and maintenance cost of the photovoltaic panel; h gb,t Output power representing a period t of the gas boiler; k gb The operating and maintenance costs of the gas boiler; h hr,t Output power, K, representing the t period of the waste heat recovery device hr Representing the operating and maintenance costs of the waste heat recovery device.
CN202110022857.3A 2021-01-08 2021-01-08 Industrial microgrid load optimization scheduling method and system containing cogeneration Active CN112821465B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110022857.3A CN112821465B (en) 2021-01-08 2021-01-08 Industrial microgrid load optimization scheduling method and system containing cogeneration

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110022857.3A CN112821465B (en) 2021-01-08 2021-01-08 Industrial microgrid load optimization scheduling method and system containing cogeneration

Publications (2)

Publication Number Publication Date
CN112821465A CN112821465A (en) 2021-05-18
CN112821465B true CN112821465B (en) 2022-11-29

Family

ID=75868638

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110022857.3A Active CN112821465B (en) 2021-01-08 2021-01-08 Industrial microgrid load optimization scheduling method and system containing cogeneration

Country Status (1)

Country Link
CN (1) CN112821465B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113240183B (en) * 2021-05-20 2023-08-18 西安图迹信息科技有限公司 Commercial building electric heating load day-ahead optimal scheduling method and system
CN113222297A (en) * 2021-06-08 2021-08-06 上海交通大学 Method, system, equipment and medium suitable for cyclic updating planning of solid waste base garden
CN113807564B (en) * 2021-07-28 2023-08-04 合肥工业大学 Park microgrid load optimal scheduling method and system based on two-stage reinforcement learning
CN113988356B (en) * 2021-09-02 2024-08-02 华北电力大学 DQN-based 5G fusion intelligent power distribution network energy management method
CN115065078B (en) * 2022-05-06 2023-07-04 合肥工业大学 Energy storage capacity configuration method and system in micro-grid environment
CN115411776B (en) * 2022-09-27 2023-11-14 国网江苏省电力有限公司常州供电分公司 Thermoelectric collaborative scheduling method and device for residence comprehensive energy system
CN116280140B (en) * 2023-04-13 2023-10-10 广东海洋大学 Ship hybrid power energy management method, equipment and medium based on deep learning

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018140945A1 (en) * 2017-01-30 2018-08-02 Kavehpour Hossein Pirouz Storage-combined cold, heat and power
CN106849190B (en) * 2017-03-21 2019-05-17 国网天津市电力公司 A kind of microgrid real-time scheduling method of providing multiple forms of energy to complement each other based on Rollout algorithm
CN111144728B (en) * 2019-12-18 2023-08-04 东南大学 Deep reinforcement learning-based economic dispatching method for cogeneration system
CN112084680B (en) * 2020-09-02 2023-12-26 沈阳工程学院 Energy internet optimization strategy method based on DQN algorithm

Also Published As

Publication number Publication date
CN112821465A (en) 2021-05-18

Similar Documents

Publication Publication Date Title
CN112821465B (en) Industrial microgrid load optimization scheduling method and system containing cogeneration
Fontenot et al. Modeling and control of building-integrated microgrids for optimal energy management–a review
Yan et al. Two-stage adjustable robust optimal dispatching model for multi-energy virtual power plant considering multiple uncertainties and carbon trading
Wang et al. Demand response comprehensive incentive mechanism-based multi-time scale optimization scheduling for park integrated energy system
Wang et al. A bargaining game-based profit allocation method for the wind-hydrogen-storage combined system
Chen et al. Research on day-ahead transactions between multi-microgrid based on cooperative game model
Salari et al. Fuzzy Q-learning-based approach for real-time energy management of home microgrids using cooperative multi-agent system
CN114611772B (en) Multi-agent reinforcement learning-based multi-microgrid system collaborative optimization method
Liu et al. Deep reinforcement learning based energy storage management strategy considering prediction intervals of wind power
CN113344249B (en) Block chain-based cooling, heating and power combined supply multi-microgrid optimal scheduling method and system
CN116014715A (en) Digital twinning-based comprehensive energy control method and system
Zhou et al. Multi-objective optimization and decision making for integrated energy system using STA and fuzzy TOPSIS
CN114498769B (en) High-proportion wind-solar island micro-grid group energy scheduling method and system
CN114169916B (en) Market member quotation strategy formulation method suitable for novel power system
CN117726143B (en) Environment-friendly micro-grid optimal scheduling method and system based on deep reinforcement learning
Bartels et al. Influence of hydrogen on grid investments for smart microgrids
CN113988567B (en) Energy storage sharing control method and control system thereof
CN112510690B (en) Optimal scheduling method and system considering wind-fire-storage combination and demand response reward and punishment
CN113807564A (en) Garden micro-grid load optimization scheduling method and system based on two-stage reinforcement learning
Wang et al. Multi-time scale optimization study of integrated energy system considering dynamic energy hub and dual demand response
An et al. Real-time optimal operation control of micro energy grid coupling with electricity-thermal-gas considering prosumer characteristics
Hu et al. Distributed multi-energy trading for energy Internet: An aggregative game approach
Tong et al. Economical optimization for multi-microgrid based on Stackelberg game
Bousnina et al. Deep Reinforcement Learning for optimal energy management of multi-energy Smart Grids
Gong et al. Economic dispatching strategy of double lead-acid battery packs considering various factors

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant