CN112290535A - Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization - Google Patents

Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization Download PDF

Info

Publication number
CN112290535A
CN112290535A CN202011008104.9A CN202011008104A CN112290535A CN 112290535 A CN112290535 A CN 112290535A CN 202011008104 A CN202011008104 A CN 202011008104A CN 112290535 A CN112290535 A CN 112290535A
Authority
CN
China
Prior art keywords
natural gas
energy system
action
time
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011008104.9A
Other languages
Chinese (zh)
Inventor
胡维昊
杜月芳
李坚
张斌
曹迪
黄越辉
王晓蓉
许潇
邓惠文
王浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
China Electric Power Research Institute Co Ltd CEPRI
Original Assignee
University of Electronic Science and Technology of China
China Electric Power Research Institute Co Ltd CEPRI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China, China Electric Power Research Institute Co Ltd CEPRI filed Critical University of Electronic Science and Technology of China
Priority to CN202011008104.9A priority Critical patent/CN112290535A/en
Publication of CN112290535A publication Critical patent/CN112290535A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/008—Circuit arrangements for ac mains or ac distribution networks involving trading of energy or energy transmission rights
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • G06Q10/06312—Adjustment or analysis of established resource schedule, e.g. resource or task levelling, or dynamic rescheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • G06Q10/06313—Resource planning in a project environment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Electricity, gas or water supply
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
    • H02J3/381—Dispersed generators
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00—Circuit arrangements for ac mains or ac distribution networks
    • H02J3/38—Arrangements for parallely feeding a single network by two or more generators, converters or transformers
    • H02J3/46—Controlling of the sharing of output between the generators, converters, or transformers
    • H02J3/466—Scheduling the operation of the generators, e.g. connecting or disconnecting generators to meet a given demand
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
    • H02J2203/10—Power transmission or distribution systems management focussing at grid-level, e.g. load flow analysis, node profile computation, meshed network optimisation, active network management or spinning reserve management
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2203/00—Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
    • H02J2203/20—Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
    • H02J2300/20—The dispersed energy generation being of renewable origin
    • H02J2300/28—The renewable source being wind energy
    • H—ELECTRICITY
    • H02—GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02J—CIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2300/00—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation
    • H02J2300/40—Systems for supplying or distributing electric power characterised by decentralized, dispersed, or local generation wherein a plurality of decentralised, dispersed or local energy generation technologies are operated simultaneously
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E10/00—Energy generation through renewable energy sources
    • Y02E10/70—Wind energy
    • Y02E10/76—Power conversion electric or electronic aspects
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E40/00—Technologies for an efficient electrical power generation, transmission or distribution
    • Y02E40/70—Smart grids as climate change mitigation technology in the energy generation sector
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

The invention discloses an online scheduling method of an electric-natural gas integrated energy system based on a depth certainty gradient strategy, which aims at the double uncertainties of intermittence of wind power, randomness of real-time electric power and natural gas markets and the like, simultaneously considers the net load fluctuation of an electric power system and the economic income of the system, adopts a depth reinforcement learning method, converts the real-time income model solving process of the electric-natural gas integrated energy system into a limited Markov decision process, and adopts a depth certainty gradient strategy (DDPG) algorithm to solve the decision problem; the electric-natural gas comprehensive energy system has high stability and economy because net load fluctuation is considered in the optimized operation of the electric-natural gas comprehensive energy system, and wind power is smoothly connected into a power grid.

Description

Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization
Technical Field
The invention belongs to the technical field of new energy power generation, and particularly relates to an on-line scheduling method of an electricity-natural gas comprehensive energy system based on a depth certainty gradient strategy.
Background
The comprehensive energy system is expected to solve the contradiction of renewable energy penetration and is developed into a new mode of future energy utilization. Natural gas is used as an important primary energy source and is most closely related to a power grid. Therefore, the electricity-natural gas integrated energy system has become the basis of the energy internet.
The traditional energy management method has deterministic rules and abstract models. Both have limitations respectively: deterministic rules can only run optimally in stationary systems and are limited by fixed variables; the process of creating an abstract model is complex and generally depends on creating an actual model, the performance of which is related to the skill and experience of a modeler. In addition, due to the intermittent characteristic of wind power, the operation risk of a power grid containing high-permeability wind power is high, the emergency reserve capacity is large, and the stability and the economy of the power grid are not guaranteed. In the existing optimization scheduling research of the electricity-natural gas integrated energy system, wind power is mostly scheduled based on day-ahead optimization, and because wind power generation has a fluctuation characteristic which is difficult to predict, the method is difficult to ensure the real-time optimal operation of the photovoltaic system.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides an on-line scheduling method of an electric-natural gas integrated energy system based on a depth certainty gradient strategy, which aims to minimize the net load fluctuation while considering the lowest system operation cost and realizes the real-time optimized operation of the electric-natural gas integrated energy system through a depth certainty gradient strategy algorithm.
In order to achieve the aim, the invention provides an on-line scheduling method of an electricity-natural gas integrated energy system based on a depth deterministic gradient strategy, which comprises the following steps:
(1) collecting historical power generation data of the wind power station wp and recording the historical power generation data as Pwp,tWherein t represents time, t is 1,2,3, …; collecting historical on-line electricity price of the electricity-natural gas comprehensive energy system and recording as lambdat(ii) a Market price of collected natural gas, recorded as kappat(ii) a Collecting the user load of the integrated energy system of electric-natural gas, and recording as Pl,t；
(2) Constructing a power output model of each component of the electricity-natural gas integrated energy system (IEngs);
(2.1) converting electricity into gas p2g equipment output model;
Figure BDA0002696650960000021
wherein the content of the first and second substances,
Figure BDA0002696650960000022
for the conversion efficiency of the P2g plant, Pp2g,tFor the power consumed by the p2g device at time t, HgIs the heating value of natural gas, Qp2g,tThe output natural gas flow rate of the plant at time t, p2 g;
(2.2) a gas turbine gt equipment output model;
Figure BDA0002696650960000023
wherein the content of the first and second substances,
Figure BDA0002696650960000024
for the conversion efficiency of the plant, Pgt,tElectric energy output for gt device at time t, Qgt,tThe natural gas flow consumed by the equipment at the time t;
(2.3) a thermal power generating unit g equipment output model;
Pg,t＝Pp2g,t+Pl,t-αtPwp,t-Pgt,t
wherein, Pg,tFor the output electric power of g equipment of the thermal power generating unit, Pl,tFor the consumer electrical load at time t, αtThe ratio of wind power to power grid at the moment t;
(3) constructing an online scheduling objective function and constraint conditions of the electricity-natural gas integrated energy system;
the objective function is:
Figure BDA0002696650960000025
wherein f isoc,tFor the operating cost of the electric-natural gas integrated energy system at time t, f (p)g,t) Expressed as the running cost of the thermal power generating unit at the moment t, { Cn,Ce,Cwp,Cp2g,CgtThe cost coefficients are respectively a supply cost coefficient of a natural gas source, an air outlet cost coefficient of an air storage tank, a cost coefficient of a wind power access power grid, an operation cost coefficient of p2g, an operation cost coefficient of gt, and Qn,tThe time t is the gas output of the natural gas source, when Q isn,tLess than 0, which means that the natural gas source gives out gas to the natural gas market at the time t to obtain profit when Q isn,tGreater than 0, which indicates that the natural gas source is changed to electricity-natural gas at the time tThe gas system supplies natural gas; pnet,tExpressed as the net load of the electro-natural gas system at time t, omega is the economic reduction factor, fpls,tEconomic losses expressed as net load fluctuations adjacent in time t; { ag,bg,cgThe coefficient is the operation cost coefficient of the thermal power generating unit; t is an optimized operation period, and F is the operation cost and the net load fluctuation economic conversion cost of the electricity-natural gas integrated energy system in the operation period;
the constraint conditions are as follows:
Figure BDA0002696650960000031
wherein the content of the first and second substances,
Figure BDA0002696650960000032
represents the minimum gas output of the natural gas source,
Figure BDA0002696650960000033
represents the maximum gas output of the natural gas source,
Figure BDA0002696650960000034
indicating the maximum amount of intake air of the air tank,
Figure BDA0002696650960000035
indicating the minimum amount of intake air of the air tank,
Figure BDA0002696650960000036
which indicates the maximum amount of intake air of the gas turbine,
Figure BDA0002696650960000037
represents the maximum electrical energy input for electrical conversion,
Figure BDA0002696650960000038
the minimum output of the thermal power generating unit is shown,
Figure BDA0002696650960000039
representing the maximum output of the thermal power generating unit;
(4) establishing and training an electric-natural gas integrated energy system online scheduling model based on a Deep Deterministic Gradient Policy (DDPG);
(4.1) converting an online scheduling objective function of the electric-natural gas integrated energy system in one operation period into a Markov decision process comprising a state set S, an action set A and a reward function r;
wherein S comprises all states in the decision process, and the state S at the moment tt＝{Pwp,t,λt,κt,SoCt,Pl.t}; a comprises all actions of the decision process, and the action at the moment t
Figure BDA00026966509600000310
At time t at stLower execution atThe real-time reward obtained is denoted as rt(at|st)；
rt(at|st)＝-(foc,t+fpls,t)
(4.2) four groups of neural networks with the same structure required by the DDPG algorithm are constructed;
constructing two action networks on line, recording the two action networks as mu and mu', wherein parameter sets are respectively recorded as thetaμ、θμ'For realizing an input state stTo the output action at；
Two evaluation networks are constructed on line and are recorded as upsilon and upsilon', and parameter sets are recorded as theta respectivelyυ、θυ'For realizing an input state stAnd an output action atTo the action merit function Qπ(st,at) Pi is a mapping strategy;
(4.3) setting the total iteration number N of an electric-natural gas integrated energy system online scheduling model based on a depth certainty gradient strategy algorithm and the iteration step number T of a Markov process; setting a memory bank, recording the capacity of the memory bank as M, and initializing the memory bank to be empty; initializing parameter sets of all neural networks, initializing n to 1, initializing a learning rate alpha and initializing a counter m;
(4.4) resetting the electricity-natural gas comprehensive energy system, setting t to 1 and acquiring the current state stThen a Markov process is performed once;
(4.5) judging whether T is smaller than T, if T is smaller than T, entering the step (4.6), otherwise, entering the step (4.14);
(4.6) mixing stInput to the action network mu to obtain the output action at；
(4.7) operation a according to the outputtCalculating the objective function value r in the step (4.1)tWhile obtaining atState s at the next moment after the actiont+1；
(4.8) construction of tuple information st,at,rt,st+1Storing the position M% M in a memory bank, and then assigning M to be M + 1;
(4.9) judging whether M is larger than M, if so, entering the step (4.10); otherwise, entering the step (4.13);
(4.10) updating the parameter set theta of the action network mu on line based on the depth deterministic gradient strategyμ；
Figure BDA0002696650960000041
Figure BDA0002696650960000042
Where b represents the number of tuple information with equal probability sampling put back from the memory bank, si,aiRepresenting the corresponding state and action in the ith tuple information;
Figure BDA0002696650960000043
is shown in state siAnd action aiCalculating a for the action cost functioniThe gradient of (a) of (b) is,
Figure BDA0002696650960000044
is shown in state siOf lower network muA gradient;
(4.11) updating the parameter set theta of the evaluation network upsilon on line by minimizing a loss functionυ；
Figure BDA0002696650960000045
Figure BDA0002696650960000046
Wherein, L (theta)υ) A loss function of the network v; q(s)i,ai) Is shown in state siAnd action aiEvaluating the action value function value output by the network upsilon; y isiAn estimate representing the υ' network; gamma is a discount factor that is a function of,
Figure BDA0002696650960000051
a gradient representing a loss function of the network v;
(4.12), updating a parameter set of the action network mu 'and the evaluation network upsilon';
Figure BDA0002696650960000052
wherein tau is an update coefficient;
(4.13) updating the current state, assigning t as t +1 and st＝st+1And then returning to the step (4.7);
(4.14), making N equal to N +1, judging whether N is larger than N, if yes, entering the step (4.15); otherwise, entering the step (4.4);
(4.15) stopping iteration, and outputting a neural network parameter set to obtain an on-line scheduling model of the electricity-natural gas integrated energy system;
(5) real-time collection of photovoltaic output
Figure BDA0002696650960000053
Electricity price data
Figure BDA0002696650960000054
Natural gas price data
Figure BDA0002696650960000055
And user load
Figure BDA0002696650960000056
Then determining the real time
Figure BDA0002696650960000057
And constructs the real-time state at the moment t
Figure BDA0002696650960000058
Finally will be
Figure BDA0002696650960000059
Inputting the data into an on-line scheduling model of the electricity-natural gas integrated energy system to obtain real-time output action
Figure BDA00026966509600000510
Then follow
Figure BDA00026966509600000511
And the on-line scheduling of the electricity-natural gas comprehensive energy system is realized.
The invention aims to realize the following steps:
the invention relates to an on-line scheduling method of an electric-natural gas integrated energy system based on a depth certainty gradient strategy, which adopts a depth reinforcement learning method to convert a real-time income model solving process of the electric-natural gas integrated energy system into a limited Markov decision process and adopts a depth certainty gradient strategy (DDPG) algorithm to solve the decision problem aiming at the double uncertainties of intermittence of wind power, randomness of real-time electric power and natural gas markets and the like and simultaneously considering the net load fluctuation of the electric power system and the economic income of the system; the electric-natural gas comprehensive energy system has high stability and economy because net load fluctuation is considered in the optimized operation of the electric-natural gas comprehensive energy system, and wind power is smoothly connected into a power grid.
Meanwhile, the on-line scheduling method of the electricity-natural gas integrated energy system based on the depth certainty gradient strategy also has the following beneficial effects:
(1) compared with the traditional excellent Particle Swarm Optimization (PSO) and deep Q network, the result shows that the DDPG algorithm-based online scheduling method for the electric-natural gas integrated energy system has the advantages of short time consumption, good optimization result and the like, so that the DDPG algorithm has better superiority in coping with various uncertain factors in the system.
(2) The invention considers the net load fluctuation during the optimized operation, and enables the wind power to be smoothly connected into the power grid, thereby ensuring the real-time optimal operation of the electricity-natural gas comprehensive energy system and having higher stability and economy.
Drawings
FIG. 1 is a flow chart of an online scheduling method of an electric-natural gas integrated energy system based on a depth deterministic gradient strategy;
FIG. 2 is a block diagram of an electric-natural gas integrated energy system;
FIG. 3 is a training flow diagram of an electric-natural gas integrated energy system online scheduling model;
FIG. 4 is a graph of the output of various components based on four scenarios;
Detailed Description
The following description of the embodiments of the present invention is provided in order to better understand the present invention for those skilled in the art with reference to the accompanying drawings. It is to be expressly noted that in the following description, a detailed description of known functions and designs will be omitted when it may obscure the subject matter of the present invention.
Examples
In the present embodiment, as shown in fig. 2, the electricity-natural gas integrated energy system is composed of a wind farm, a power grid and a natural gas network;
the wind power plant is connected with the bus bar through a transformer; the busbar is connected to the grid via an overhead transmission line.
The method for optimizing the electricity-natural gas integrated energy system in real time based on the depth deterministic gradient strategy is described in detail with reference to fig. 2.
In this embodiment, as shown in fig. 1, the method for online scheduling of an electricity-natural gas integrated energy system based on a depth deterministic gradient strategy of the present invention includes the following steps:
s1, collecting historical power generation data of the wind power station wp in the past year and recording the historical power generation data as Pwp,tWherein t represents a time, and t is 1,2, 3.; the historical online electricity price of the past year of the electricity-natural gas comprehensive energy system is collected and recorded as lambdat(ii) a The price of the collected natural gas market over the past year is recorded as kappat(ii) a Collecting the user load of the past year of the electric-natural gas integrated energy system, and recording as Pl,t；
S2, constructing output models of all components of the electricity-natural gas integrated energy system (IEngs);
Figure BDA0002696650960000071
wherein the content of the first and second substances,
Figure BDA0002696650960000072
the conversion efficiency of the P2g equipment is 0.8, Pp2g,tFor the power consumed by the p2g device at time t, HgIs the heating value of natural gas, Qp2g,tThe output natural gas flow rate of the plant at time t, p2 g;
Figure BDA0002696650960000073
the conversion efficiency of the gt device is taken as 0.8, Pgt,tElectric energy output for gt device at time t, Qgt,tThe natural gas flow consumed by the equipment at the time t; pg,tFor the output electric power of g equipment of the thermal power generating unit, Pl,tFor the consumer electrical load at time t, αtThe value range of the ratio of the wind power to the power grid at the moment t is [0,1 ]]；
S3, constructing an online scheduling objective function and constraint conditions of the electricity-natural gas integrated energy system;
the objective function is:
Figure BDA0002696650960000074
wherein f isoc,tFor the operating cost of the electric-natural gas integrated energy system at time t, f (p)g,t) Expressed as the running cost of the thermal power generating unit at the moment t, { Cn,Ce,Cwp,Cp2g,CgtThe values of the supply cost coefficient of the natural gas source, the air outlet cost coefficient of the air storage tank, the abandoned wind cost coefficient, the operation cost coefficient of p2g and the operation cost coefficient of gt are shown in a table 1, and Q is shown in the tablen,tThe time t is the gas output of the natural gas source, when Q isn,tLess than 0, which means that the natural gas source gives out gas to the natural gas market at the time t to obtain profit when Q isn,tWhen the natural gas source is more than 0, the natural gas source supplies natural gas to the gas inlet of the electric-natural gas system at the time t; pnet,tExpressed as the net load of the electro-natural gas system at time t, omega is the economic reduction factor, fpls,tEconomic losses expressed as net load fluctuations adjacent in time t; { ag,bg,cgThe coefficient is the operation cost coefficient of the thermal power generating unit; t is an optimized operation period, and F is the operation cost and the net load fluctuation economic conversion cost of the electricity-natural gas integrated energy system in the operation period;
Cn Ce Cwp Cp2g Cgt
30($/m3) 20($/m3) 80($/MW) 200($/MW) 200($/MW)
TABLE 1
The constraint conditions are as follows:
Figure BDA0002696650960000081
wherein the content of the first and second substances,
Figure BDA0002696650960000082
the minimum gas output of the natural gas source is shown and is 10m3，
Figure BDA0002696650960000083
The maximum gas output of the natural gas source is represented and is 500m3，
Figure BDA0002696650960000084
The maximum air inflow of the air storage tank is represented and is 300m3，
Figure BDA0002696650960000085
The minimum air inflow of the air storage tank is represented and is 50m3，
Figure BDA0002696650960000086
The maximum air inflow of the gas turbine is shown and is 200m3，
Figure BDA0002696650960000087
Represents the maximum electric energy input of the electric conversion gas, takes the value of 10MW,
Figure BDA0002696650960000088
indicating minimum discharge of thermal power generating unitThe force, which takes the value of 1MW,
Figure BDA0002696650960000089
the maximum output of the thermal power generating unit is represented, and the value is 50 MW;
s4, as shown in FIG. 3, building and training an electric-natural gas integrated energy system online scheduling model based on a Deep Deterministic Gradient Policy (DDPG);
s4.1, converting an online scheduling objective function of the electricity-natural gas integrated energy system in an operation period into a Markov decision process comprising a state set S, an action set A and a reward function r;
wherein S comprises all states in the decision process, and the state S at the moment tt＝{Pwp,t,λt,κt,SoCt,Pl.t}; a comprises all actions of the decision process, and the action at the moment t
Figure BDA00026966509600000810
At time t at stLower execution atThe real-time reward obtained is denoted as rt(at|st)；
rt(at|st)＝-(foc,t+fpls,t)
S4.2, constructing four groups of neural networks with the same structure required by the DDPG algorithm;
constructing two action networks on line, recording the two action networks as mu and mu', wherein parameter sets are respectively recorded as thetaμ、θμ'For realizing an input state stTo the output action at；
Two evaluation networks are constructed on line and are recorded as upsilon and upsilon', and parameter sets are recorded as theta respectivelyυ、θυ'For realizing an input state stAnd an output action atTo the action merit function Qπ(st,at) Pi is a mapping strategy;
s4.3, setting the total iteration number N of an electric-natural gas integrated energy system online scheduling model based on a depth certainty gradient strategy algorithm to 10000 and the iteration step number T of a Markov process to 24; setting a memory bank, recording the capacity of the memory bank as M40000, and initializing the memory bank to be empty; initializing parameter sets of all neural networks, wherein n is 1, learning rate alpha is 0.0001, and a counter m is 0;
s4.4, judging whether T is smaller than T, if T is smaller than T, entering the step S4.5, otherwise, entering the step S4.13;
s4.5, mixing StInput to the action network mu to obtain the output action at；
S4.6, according to the output action atThe value of the objective function r in step S4.1 is calculatedtWhile obtaining atState s at the next moment after the actiont+1；
S4.7, establishing tuple information St,at,rt,st+1Storing the position M% M in a memory bank, and then assigning M to be M + 1;
s4.8, judging whether M is larger than M, if so, entering a step S4.9; otherwise, the step S4.4 is carried out;
s4.9, updating parameter set theta of action network mu on line based on depth certainty gradient strategyμ；
Figure BDA0002696650960000091
Figure BDA0002696650960000092
Where b represents the number of tuple information with equal probability sampling put back from the memory bank, si,aiRepresenting the corresponding state and action in the ith tuple information;
Figure BDA0002696650960000093
is shown in state siAnd action aiCalculating a for the action cost functioniThe gradient of (a) of (b) is,
Figure BDA0002696650960000094
is shown in state siGradient of lower network μ;
s4.10, updating the parameter set theta of the evaluation network upsilon on line through a minimum loss functionυ；
Figure BDA0002696650960000095
Figure BDA0002696650960000096
Wherein, L (theta)υ) A loss function of the network v; q(s)i,ai) Is shown in state siAnd action aiEvaluating the action value function value output by the network upsilon; y isiAn estimate representing the υ' network; gamma is a discount factor that is a function of,
Figure BDA0002696650960000097
a gradient representing a loss function of the network v;
s4.11, updating a parameter set of the action network mu 'and the evaluation network upsilon';
Figure BDA0002696650960000098
wherein tau is an update coefficient and takes a value of 0.01;
s4.12, updating the current state, assigning t as t +1 and St＝st+1Then returning to step S4.7;
s4.13, let N be N +1, and then determine whether N is greater than N, if yes, go to step S4.14; otherwise, the step S4.3 is carried out;
s4.14, stopping iteration, and outputting a neural network parameter set to obtain an online scheduling model of the electricity-natural gas integrated energy system;
s5, collecting photovoltaic output in real time
Figure BDA0002696650960000101
Electricity price data
Figure BDA0002696650960000102
Natural gas price data
Figure BDA0002696650960000103
And user load
Figure BDA0002696650960000104
Then determining the real time
Figure BDA0002696650960000105
And constructs the real-time state at the moment t
Figure BDA0002696650960000106
Finally will be
Figure BDA0002696650960000107
Inputting the data into an on-line scheduling model of the electricity-natural gas integrated energy system to obtain real-time output action
Figure BDA0002696650960000108
Then follow
Figure BDA0002696650960000109
And the on-line scheduling of the electricity-natural gas comprehensive energy system is realized.
In this embodiment, the processor is selected as an Inter (R) core (TM) i9-9820X CPU @3.30GHz hardware platform, and Python3.7 and tensoflow1.8.0 are used to implement the embodiment of the method of the present invention. And carrying out real-time test on the successfully trained neural network based on four scenes, namely the following four scenes.
Scene one: the p2g equipment and a net load fluctuation model are not contained in the electricity-natural gas integrated energy system, and the optimization target only considers economic benefits;
scene two: the p2g equipment is contained in the electricity-natural gas integrated energy system, and the optimization goal only considers the economic benefit;
scene three: the electric-natural gas integrated energy system comprises a net load fluctuation model, and the optimization target simultaneously considers economic benefit and net load fluctuation to convert economic cost;
scene four: the electric-natural gas integrated energy system simultaneously comprises p2g equipment and a net load fluctuation model, and the optimization objective simultaneously considers economic benefit and net load fluctuation to reduce economic cost;
the result shows that as shown in fig. 4, it can be seen in the figure that when a p2g device is introduced into the electricity-natural gas integrated energy system, a large amount of wind power is absorbed, wind curtailment is obviously reduced, which is beneficial to increase economic benefit, meanwhile, since a net load fluctuation model is not considered, the net load curve fluctuation is obvious, the stability of the power grid is reduced, but the fluctuation amplitude of the net load fluctuation is obviously reduced compared with the net load fluctuation when the p2g device is not introduced, which indicates that p2g plays a role in suppressing the net load fluctuation. When the p2g equipment and the net load fluctuation model are introduced simultaneously, the economic benefit and the stability of the electricity-natural gas integrated energy system are both considered. In addition, in order to further embody the superiority of the method of the present invention, in the embodiment, comparison is made with a conventional particle swarm optimization algorithm (PSO) and a deep Q network algorithm (DQN). The results show that: the DDPG method is a method provided by the invention, and because a large amount of simulation training is carried out on historical wind power output, namely, a neural network describes the output probability characteristic of the wind power, real-time scheduling can be realized; the PSO method is used for carrying out day-ahead optimal economic dispatching according to predicted wind power output, electricity price data and natural gas price data when the economic dispatching of the electricity-natural gas comprehensive energy system is realized, and the day-ahead dispatching strictly depends on the accuracy of a data predicted value; the traditional DQN algorithm neural network is simple in structure and short in time consumption, but the actions need to be discretized, all possible actions in all systems are not considered, and the comprehensiveness of the optimization effect is not enough.
Table 2 shows the comparison of the different methods;
method of producing a composite material Average running cost ($/day) Calculating elapsed time (seconds)
DQN 21453 32.2
PSO 26883 37812
DDPG 18724 62.3
TABLE 2
Although illustrative embodiments of the present invention have been described above to facilitate the understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the embodiments, and various changes may be made apparent to those skilled in the art as long as they are within the spirit and scope of the present invention as defined and defined by the appended claims, and all matters of the invention which utilize the inventive concepts are protected.

Claims (1)

1. An online scheduling method of an electricity-gas integrated energy system based on deep strategy optimization comprises the following steps:
(1) collecting historical power generation data of the wind power station wp and recording the historical power generation data as Pwp,tWherein t represents time, t is 1,2,3, …; collecting historical on-line electricity price of the electricity-natural gas comprehensive energy system and recording as lambdat(ii) a Market price of collected natural gas, recorded as kappat(ii) a Collecting the user load of the integrated energy system of electric-natural gas, and recording as Pl,t；
(2) Constructing an output model of each component of the electricity-natural gas integrated energy system;
(2.1) converting electricity into gas p2g equipment output model;
Figure FDA0002696650950000011
wherein the content of the first and second substances,
Figure FDA0002696650950000012
for the conversion efficiency of the P2g plant, Pp2g,tFor the power consumed by the p2g device at time t, HgIs the heating value of natural gas, Qp2g,tThe output natural gas flow rate of the plant at time t, p2 g;
(2.2) a gas turbine gt equipment output mode model;
Figure FDA0002696650950000013
wherein, Pgt,tElectric energy output for gt device at time t, Qgt,tThe natural gas flow consumed by the equipment at the time t;
(2.3) a thermal power generating unit g equipment output model;
Pg,t＝Pp2g,t+Pl,t-αtPwp,t-Pgt,t
wherein, Pg,tFor the output electric power of g equipment of the thermal power generating unit, Pl,tFor the consumer electrical load at time t, αtThe ratio of wind power to power grid at the moment t;
(3) constructing an online scheduling objective function and constraint conditions of the electricity-natural gas integrated energy system;
the objective function is:
Figure FDA0002696650950000014
wherein f isoc,tFor the operating cost of the electric-natural gas integrated energy system at time t, f (p)g,t) Expressed as the running cost of the thermal power generating unit at the moment t, { Cn,Ce,Cwp,Cp2g,CgtThe cost coefficients are respectively a supply cost coefficient of a natural gas source, an air outlet cost coefficient of an air storage tank, a cost coefficient of a wind power access power grid, an operation cost coefficient of p2g, an operation cost coefficient of gt, and Qn,tThe time t is the gas output of the natural gas source, when Q isn,tLess than 0, which means that the natural gas source gives out gas to the natural gas market at the time t to obtain profit when Q isn,tWhen the natural gas source is more than 0, the natural gas source supplies natural gas to the gas inlet of the electric-natural gas system at the time t; pnet,tExpressed as the net load of the electro-natural gas system at time t, omega is the economic reduction factor, fpls,tEconomic losses expressed as net load fluctuations adjacent in time t; { ag,bg,cgThe coefficient is the operation cost coefficient of the thermal power generating unit; t is an optimized operation period, and F is the operation cost and the net load fluctuation economic conversion cost of the electricity-natural gas integrated energy system in the operation period;
the constraint conditions are as follows:
Figure FDA0002696650950000021
wherein the content of the first and second substances,
Figure FDA0002696650950000022
represents the minimum gas output of the natural gas source,
Figure FDA0002696650950000023
represents the maximum gas output of the natural gas source,
Figure FDA0002696650950000024
indicating the maximum amount of intake air of the air tank,
Figure FDA0002696650950000025
indicating the minimum amount of intake air of the air tank,
Figure FDA0002696650950000026
which indicates the maximum amount of intake air of the gas turbine,
Figure FDA0002696650950000027
represents the maximum electrical energy input for electrical conversion,
Figure FDA0002696650950000028
the minimum output of the thermal power generating unit is shown,
Figure FDA0002696650950000029
representing the maximum output of the thermal power generating unit;
(4) establishing and training an electric-natural gas integrated energy system online scheduling model based on a Deep Deterministic Gradient Policy (DDPG);
(4.1) converting an online scheduling objective function of the electric-natural gas integrated energy system in one operation period into a Markov decision process comprising a state set S, an action set A and a reward function r;
wherein S comprises all states in the decision process, and the state S at the moment tt＝{Pwp,t,λt,κt,SoCt,Pl.t}; a comprises all actions of the decision process, and the action at the moment t
Figure FDA00026966509500000210
At time t at stLower execution atThe real-time reward obtained is denoted as rt(at|st)；
rt(at|st)＝-(foc,t+fpls,t)
(4.2) four groups of neural networks with the same structure required by the DDPG algorithm are constructed;
constructing two action networks on line, recording the two action networks as mu and mu', wherein parameter sets are respectively recorded as thetaμ、θμ'For realizing an input state stTo the output action at；
Two evaluation networks are constructed on line and are recorded as upsilon and upsilon', and parameter sets are recorded as theta respectivelyυ、θυ'For realizing an input state stAnd an output action atTo the action merit function Qπ(st,at) Pi is a mapping strategy;
(4.3) setting the total iteration number N of an electric-natural gas integrated energy system online scheduling model based on a depth certainty gradient strategy algorithm and the iteration step number T of a Markov process; setting a memory bank, recording the capacity of the memory bank as M, and initializing the memory bank to be empty; initializing parameter sets of all neural networks, initializing n to 1, initializing a learning rate alpha and initializing a counter m;
(4.4) resetting the electricity-natural gas comprehensive energy system, setting t to 1 and acquiring the current state stThen a Markov process is performed once;
(4.5) judging whether T is smaller than T, if T is smaller than T, entering the step (4.6), otherwise, entering the step (4.14);
(4.6) mixing stInput to the action network mu to obtain the output action at；
(4.7) operation a according to the outputtCalculating the objective function value r in the step (4.1)tWhile obtaining atState s at the next moment after the actiont+1；
(4.8) construction of tuple information st,at,rt,st+1Storing the position M% M in a memory bank, and then assigning M to be M + 1;
(4.9) judging whether M is larger than M, if so, entering the step (4.10); otherwise, entering the step (4.13);
(4.10) updating the parameter set theta of the action network mu on line based on the depth deterministic gradient strategyμ；
Figure FDA0002696650950000031
Figure FDA0002696650950000032
Where b represents the number of tuple information with equal probability sampling put back from the memory bank, si,aiRepresenting the corresponding state and action in the ith tuple information;
Figure FDA0002696650950000033
is shown in state siAnd action aiCalculating a for the action cost functioniThe gradient of (a) of (b) is,
Figure FDA0002696650950000034
is shown in state siGradient of lower network μ;
(4.11) updating the parameter set theta of the evaluation network upsilon on line by minimizing a loss functionυ；
Figure FDA0002696650950000041
Figure FDA0002696650950000042
Wherein, L (theta)υ) A loss function of the network v; q(s)i,ai) Is shown in state siAnd action aiEvaluating the action value function value output by the network upsilon; y isiAn estimate representing the υ' network; gamma is a discount factor that is a function of,
Figure FDA0002696650950000043
a gradient representing a loss function of the network v;
(4.12), updating a parameter set of the action network mu 'and the evaluation network upsilon';
Figure FDA0002696650950000044
wherein tau is an update coefficient;
(4.13) updating the current state, assigning t as t +1 and st＝st+1And then returning to the step (4.7);
(4.14), making N equal to N +1, judging whether N is larger than N, if yes, entering the step (4.15); otherwise, entering the step (4.4);
(4.15) stopping iteration, and outputting a neural network parameter set to obtain an on-line scheduling model of the electricity-natural gas integrated energy system;
(5) real-time collection of photovoltaic output
Figure FDA0002696650950000045
Electricity price data
Figure FDA0002696650950000046
Natural gas price data
Figure FDA0002696650950000047
And user load
Figure FDA0002696650950000048
Then determining the real time
Figure FDA0002696650950000049
And constructs the real-time state at the moment t
Figure FDA00026966509500000410
Finally will be
Figure FDA00026966509500000411
Inputting the data into an on-line scheduling model of the electricity-natural gas integrated energy system to obtain real-time output action
Figure FDA00026966509500000412
Then follow
Figure FDA00026966509500000413
And the on-line scheduling of the electricity-natural gas comprehensive energy system is realized.
CN202011008104.9A 2020-09-23 2020-09-23 Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization Pending CN112290535A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011008104.9A CN112290535A (en) 2020-09-23 2020-09-23 Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011008104.9A CN112290535A (en) 2020-09-23 2020-09-23 Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization

Publications (1)

Publication Number Publication Date
CN112290535A true CN112290535A (en) 2021-01-29

Family

ID=74421450

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011008104.9A Pending CN112290535A (en) 2020-09-23 2020-09-23 Online scheduling method of electricity-gas integrated energy system based on deep strategy optimization

Country Status (1)

Country Link
CN (1) CN112290535A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106208157A (en) * 2016-07-19 2016-12-07 河海大学 The electrical interconnection integrated energy system peak load shifting method of gas is turned based on electricity
CN110135631A (en) * 2019-04-26 2019-08-16 燕山大学 Electrical integrated energy system dispatching method based on information gap decision theory

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106208157A (en) * 2016-07-19 2016-12-07 河海大学 The electrical interconnection integrated energy system peak load shifting method of gas is turned based on electricity
CN110135631A (en) * 2019-04-26 2019-08-16 燕山大学 Electrical integrated energy system dispatching method based on information gap decision theory

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BIN ZHANG 等: ""Dynamic energy conversion and management strategy for an integrated electricity and natural gas system with renewable energy: Deep reinforcement learning approach"", 《ENERGY CONVERSION AND MANAGEMENT》 *
李涛 等: "基于深度强化学习算法的光伏-抽储互补系统智能调度"", 《电工技术学报》 *

Similar Documents

Publication Publication Date Title
Yan et al. Two-stage adjustable robust optimal dispatching model for multi-energy virtual power plant considering multiple uncertainties and carbon trading
CN105071389B (en) The alternating current-direct current mixing micro-capacitance sensor optimizing operation method and device of meter and source net load interaction
CN107895971A (en) Regional Energy internet dispatching method based on stochastic programming and Model Predictive Control
CN111681130B (en) Comprehensive energy system optimal scheduling method considering conditional risk value
CN103544372B (en) A kind of Unit Selection method of miniature gas turbine cogeneration system
CN112290536B (en) Online scheduling method of electricity-heat comprehensive energy system based on near-end strategy optimization
CN104065072A (en) Micro-grid operation optimization method based on dynamic electricity price
CN112952807B (en) Multi-objective optimization scheduling method considering wind power uncertainty and demand response
CN111668878A (en) Optimal configuration method and system for renewable micro-energy network
CN112072643A (en) Light-storage system online scheduling method based on depth certainty gradient strategy
CN116247648A (en) Deep reinforcement learning method for micro-grid energy scheduling under consideration of source load uncertainty
CN113592133A (en) Energy hub optimal configuration method and system
CN111181201A (en) Multi-energy park scheduling method and system based on double-layer reinforcement learning
CN114611772B (en) Multi-agent reinforcement learning-based multi-microgrid system collaborative optimization method
Wen et al. ELCC-based capacity value estimation of combined wind-storage system using IPSO algorithm
Cai et al. Optimal scheduling of a hybrid AC/DC multi-energy microgrid considering uncertainties and Stackelberg game-based integrated demand response
Wang et al. Unified operation optimization model of integrated coal mine energy systems and its solutions based on autonomous intelligence
CN113363976B (en) Scene graph-based wind-solar-water complementary power generation system medium-term optimization scheduling method
CN114169916A (en) Market member quotation strategy making method suitable for novel power system
CN112308411B (en) Comprehensive energy station random planning method and system based on dynamic carbon trading model
CN113158547B (en) Regional comprehensive energy system optimal configuration method considering economy and reliability
CN108233357A (en) Wind-powered electricity generation based on nonparametric probabilistic forecasting and risk expectation dissolves optimization method a few days ago
CN114301089A (en) Energy storage capacity configuration optimization method for wind-solar combined power generation system
Zhang et al. A double-deck deep reinforcement learning-based energy dispatch strategy for an integrated electricity and district heating system embedded with thermal inertial and operational flexibility
CN113298407A (en) Industrial park electricity-gas comprehensive energy system optimization scheduling model establishing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210129