CN107026462B - Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power - Google Patents

Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power Download PDF

Info

Publication number
CN107026462B
CN107026462B CN201710470529.3A CN201710470529A CN107026462B CN 107026462 B CN107026462 B CN 107026462B CN 201710470529 A CN201710470529 A CN 201710470529A CN 107026462 B CN107026462 B CN 107026462B
Authority
CN
China
Prior art keywords
energy storage
storage device
hybrid system
power
wind
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201710470529.3A
Other languages
Chinese (zh)
Other versions
CN107026462A (en
Inventor
李泽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
North China Electric Power University
Original Assignee
North China Electric Power University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by North China Electric Power University filed Critical North China Electric Power University
Priority to CN201710470529.3A priority Critical patent/CN107026462B/en
Publication of CN107026462A publication Critical patent/CN107026462A/en
Application granted granted Critical
Publication of CN107026462B publication Critical patent/CN107026462B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H02GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02JCIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00Circuit arrangements for ac mains or ac distribution networks
    • H02J3/28Arrangements for balancing of the load in a network by storage of energy
    • HELECTRICITY
    • H02GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02JCIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00Circuit arrangements for ac mains or ac distribution networks
    • H02J3/008Circuit arrangements for ac mains or ac distribution networks involving trading of energy or energy transmission rights
    • HELECTRICITY
    • H02GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02JCIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J3/00Circuit arrangements for ac mains or ac distribution networks
    • H02J3/24Arrangements for preventing or reducing oscillations of power in networks
    • H02J3/386
    • HELECTRICITY
    • H02GENERATION; CONVERSION OR DISTRIBUTION OF ELECTRIC POWER
    • H02JCIRCUIT ARRANGEMENTS OR SYSTEMS FOR SUPPLYING OR DISTRIBUTING ELECTRIC POWER; SYSTEMS FOR STORING ELECTRIC ENERGY
    • H02J2203/00Indexing scheme relating to details of circuit arrangements for AC mains or AC distribution networks
    • H02J2203/20Simulating, e g planning, reliability check, modelling or computer assisted design [CAD]
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02EREDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E10/00Energy generation through renewable energy sources
    • Y02E10/70Wind energy
    • Y02E10/76Power conversion electric or electronic aspects
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02EREDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
    • Y02E70/00Other energy conversion or management systems reducing GHG emissions
    • Y02E70/30Systems combining energy storage with energy generation of non-fossil origin

Landscapes

  • Engineering & Computer Science (AREA)
  • Power Engineering (AREA)
  • Supply And Distribution Of Alternating Current (AREA)
  • Charge And Discharge Circuits For Batteries Or The Like (AREA)

Abstract

A kind of energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power, its step are as follows:A. basic data is inputted;B. Q value function is initialized;Hop count t=1, the action selection times N having been carried out when c. settingn=0;D. hybrid system state in which is determined;E. control action is selected;F. hybrid system output power deviation control targe value is obtained;G. output power requirements of the energy storage device in the t periods are calculated;H. real output of the energy storage device in the t periods is calculated;I. t period hybrid system real output deviations are calculated;J. return value immediately is calculated;K. energy storage device t+1 period charging and discharging states are determined;L. t+1 period hybrid system control actions are determined;M. Q value function is modified;If study step number as defined in n. reaching, training terminate, otherwise t=t+1, Nn=Nn+ 1, go to step d.The reasonable control to energy storage device can be achieved in the present invention, advantageously reduces wind deposit expense, and reduction wind power output power fluctuates the influence to operation of power networks.

Description

Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power
Technical field
The present invention relates to the formulating method that wind stores up energy storage device control strategy in hybrid system, can improve wind storage hybrid system Reliability, self-disciplining and planned, belong to technical field of power generation.
Background technology
Wind-power electricity generation has the characteristics that technology maturation, cost are low, environmental-friendly, but its intermittent, fluctuation is transported to power grid Row scheduling brings challenge, becomes an important factor for restricting Wind Power Development.With the increase of wind-electricity integration ratio, only by grid side Regulation and control will be difficult to realize the reasonable consumption of wind-powered electricity generation, in recent years, from wind-powered electricity generation self-disciplining angle is improved, slows down wind-powered electricity generation and power grid is transported The impact that row is brought, is increasingly becoming new developing direction.Energy storage device can provide quick, two-way spare response, by energy storage with Wind-powered electricity generation composition wind storage hybrid system, is conducive to improve the controllability of output power, stationarity and planned.
Under Power Market, usually require that the unscheduled power that wind power plant real output is declared with it matches, If real output deviates planned value, bulk power grid scheduling pressure can be increased, it is therefore desirable to the behavior to deviateing unscheduled power Punished and collect corresponding expense.By quick, the two-way response ability of energy storage, wind storage hybrid system can pass through autogenous control Real output is tracked generation schedule set in advance as much as possible, reduce output deviation, to reduce wind storage hybrid system The rejection penalty paid, and how to make full use of limited energy storage resource to realize a plan power tracking to greatest extent as asking The key of topic.Due to the strong uncertainty of wind power output power, the regulation and control behavior of energy storage device in addition is subject to memory capacity, power The limitation of a variety of constraints such as limitation, discharge and recharge number so that the unscheduled power tracking control problem of hybrid system is sufficiently complex, because This is necessary to set a set of rational control strategy for the energy storage device of wind storage hybrid system, to improve wind storage hybrid system Reliability, self-disciplining and planned.
The content of the invention
It is an object of the invention to the drawback for the prior art, there is provided a kind of energy storage for the tracking of wind-powered electricity generation unscheduled power Equipment control strategy formulating method, to improve the wind storage economy of hybrid system, self-disciplining and planned.
Problem of the present invention is solved with following technical proposals:
A kind of energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power, the method is in electricity market It is theoretical based on intensified learning under background, wind storage hybrid system output power strategy, tool are formulated using SARSA learning methods Body step is as follows:
A. basic data is inputted, including:Hybrid system and all kinds of cost parameters of power grid transaction;The meter that hybrid system is declared Draw power, the spare capacity that hybrid system is declared, wind-powered electricity generation real output;Energy storage device power limit, capacity limit value, charge and discharge Electrical efficiency, exempt from hop count during punishment;SARSA learning algorithm parameters;Wherein, hybrid system and all kinds of cost parameters bags of power grid transaction Include spare electricity price, spare penalty price;SARSA learning algorithms parameter include Studying factors, discount factor, study step number and Penalty coefficient;
B. Q value function is initialized;
Hop count t=1, the action selection times N having been carried out when c. settingn=0;
D. according to energy storage device t period initial time storing electricities Es,t, wind-powered electricity generation actual power and hybrid system planned value it Poor Δ Ph,t, energy storage device charging and discharging state Ks,t, the spare capacity declared of hybrid systemDetermine residing for t period hybrid systems State st
E. according to current state st, value function Qt(s, a) selects control action at, comprise the following steps that;
1. take random number T in [0,1] sectionR
2. calculate probability threshold values TH
In formula:NnTo have been carried out acting the number of selection;NTThe total degree of selection is acted for training overall process;
3. carry out action selection:If TR> TH, selection acts a from possible action collection A at randomt;If TR≤TH, selection greediness Strategy is acted as this period, i.e.,:
F. according to action atAnd hybrid system output power deviation control targe valueOne-to-one relationship, obtain t The desired value of period hybrid system output power deviation control;
G. according to hybrid system output power deviation control targe valueCalculate energy storage device Power Control requirements
H. actual charge-discharge electric power of the energy storage device in the t periods is calculated
If the 1. control setting value of t period energy storage devicesEnergy storage device need to be verified and send powerAfterwards, if super Go out energy storage device lower bound of capacity, need to correct if if:
In formula:Es,tFor t period initial time energy storage device storing electricities;ηdFor the discharging efficiency of energy storage device;Δ t is Window duration;EsminFor energy storage device lower bound of capacity;For the nonce kept in when asking for energy storage real output.
If 2.Energy storage device absorbed power need to be verifiedAfterwards, if beyond energy storage device maximum size, if super Go out, need to correct:
In formula:ηcFor the charge efficiency of energy storage device;EsmaxFor energy storage device maximum size;
3. in addition, energy storage device real output should also meet the constraint of the active power upper limit, lower limit constraint, if exceeding has The work(upper limit of the power or lower limit, then need to correct:
In formula:PsmaxFor the energy storage device active power upper limit;PsminFor energy storage device active power lower limit;Set for energy storage Standby real output.
I. t period hybrid system real output deviations are calculated
In formula:For t period wind-powered electricity generation real outputs;For hybrid system t period unscheduled powers are declared to power grid Value;
J. t periods return value r immediately is calculatedt+1(st,at):
rt+1(st,at)=Cr(t)+Cp(t)+Ca(t)+Cc(t)
Cr(t) it is spare electricity expense, Cp(t) it is spare beyond spare caused rejection penalty is declared to actually use, count Calculating formula is respectively:
In formula:λr,tFor the spare electricity price of t periods;λpIt is spare beyond declaring spare model for hybrid system actual use When enclosing, rejection penalty coefficient of the power grid to hybrid system;Rh,tThe t period spare capacities declared for hybrid system;
Ca(t) punishment of set control targe value setting, meter are not reached for t period hybrid system Actual Control Effect of Strong Calculating formula is:
In formula:kcCorresponding penalty coefficient is not inconsistent it with actual value for energy storage device power demand values;
Cc(t) corresponding punishment is changed for t period energy storage device charge and discharges state:
In formula:kkFor the corresponding penalty coefficient of frequent discharge and recharge;Ks,tFor the continuous charge and discharge sustained periods of time number of energy storage, positive number Represent hop count during trickle charge, hop count during negative number representation continuous discharge;KsmaxHop count during to exempt from punishment, when energy storage is continuously filled Electricity or electric discharge KsmaxAfter a period, change charging, discharge condition will not pay for;
K. according to Ks,tDischarge and recharge behavior with the t periods is with new energy storage device t+1 period charging and discharging states Ks,t+1
L. t+1 periods wind storage hybrid system state in which s is identifiedt+1, t+1 period wind is determined according to action selection strategy Store up hybrid system control action at+1
M. Q value function is modified:
Qt+1(st,at)=Qt(st,at)+α[rt+1(st,at)+γQt(st+1,at+1)-Qt(st,at)]
In formula:Qt(st,at) it is by t-1 revised optimal action value function;α is Studying factors;
If study step number N as defined in n. reachingT, it is the control that this patent is formed to perform greedy strategy according to Q value function System strategy;If study step number, t=t+1, N as defined in not up ton=Nn+ 1, go to step d.
The above-mentioned energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power, the wind storage hybrid system institute The state s at placetIt is the element in wind storage hybrid system state set S, the definite method of the wind storage hybrid system state set S It is as follows:
By energy storage device capacity according toSiding-to-siding block length be divided into m section:[Esmin,Esmin+ ΔEs)、[Esmin+ΔEs,Esmin+2ΔEs)、…、[Esmax-ΔEs,Esmax];By wind-powered electricity generation actual power and hybrid system planned value Difference Δ Ph,tAccording toSiding-to-siding block length be divided into n section:N is even number;Will The spare capacity that hybrid system is declaredAccording toLength be divided into j discrete state:0、 In addition, energy storage device charging and discharging state shares k=2Ksmax+ 1 state:-Ksmax、- Ksmax+1、…、-1、0、1、…、Ksmax-1、Ksmax.Wind storage hybrid system shape is formed by the cartesian product of aforementioned four variable states State set S:
S={ s1,s2,…,sm×n×j×k}
The above-mentioned energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power, the possible action collection A's Building method is as follows:
By t period hybrid system output power deviation control targe valuesAs the action behavior of control system, if phase Neighbour controls the difference of setting value to beThe setting value is controlled to be 0、 …、Common 2b+1 possible action, uses a respectively1、a2、…、a2b+1Above-mentioned each control targe value is represented, Form possible action collection A:
A={ a1,a2,…,a2b+1}
In formula:a1,a2,…,a2b+1Above-mentioned 2b+1 control setting value is represented respectively.
The present invention is only needed according to the experience accumulation interacted with environment, you can successfully manages wind power output power wave zone The uncertainty come;The reasonable control to energy storage device is the method achieve, is conducive to improve wind storage hybrid system output power Self-disciplining, planned, reduce the spare expense of hybrid system, while reduce wind power output power fluctuation to operation of power networks tune The influence of degree.
Brief description of the drawings
Fig. 1 is the flow chart of the method for the present invention.
Each symbol is in text:Es,tFor t period initial time energy storage device storing electricities;Es,t+1Stored up for the t period end moment Can equipment storing electricity;Ks,tFor the charging and discharging state of energy storage device;KsmaxFor hop count when exempting to punish of energy storage device;For t Period wind-powered electricity generation real output;ΔPh,tFor t period wind-powered electricity generation actual powers and the difference of hybrid system planned value;For mixing The desired value of system output power deviation control;For t period hybrid system real output deviations;For energy storage device Output power requirements;To consider the energy storage device output power correction value after capacity-constrained;Exist for energy storage device The actual charge-discharge electric power of t periods;For hybrid system t period unscheduled power values are declared to power grid;Rh,tFor hybrid system Shen The t period spare capacities of report;stHybrid system state in which is stored up for t periods wind;st+1Hybrid system institute is stored up for t+1 periods wind The state at place;atHybrid system control action is stored up for t periods wind;at+1Hybrid system control action is stored up for t+1 periods wind;TRFor with Machine number;THFor probability threshold values;NnAction to have been carried out selects number;NTFor action selection total degree in default training process; ηdFor the discharging efficiency of energy storage device;Δ t is the duration of t periods;EsminFor energy storage device lower bound of capacity;ηcSet for energy storage Standby charge efficiency;EsmaxFor energy storage device maximum size;rt+1(st,at) be the t periods return value immediately;CT(t) it is the t periods The total spare expense of hybrid system;Cr(t) it is the spare electricity expense of t period hybrid systems;Cp(t) make for t period hybrid systems are actual With it is spare beyond declare it is spare caused by rejection penalty;Ca(t) do not reach for t period hybrid system Actual Control Effect of Strong set The punishment of fixed control targe value;Cc(t) corresponding punishment is changed for t period energy storage device charge and discharges state;λr,tFor the t periods Spare electricity price;λpExceed corresponding penalty coefficient when declaring spare for hybrid system output power;kcFor energy storage device power Requirements are not inconsistent corresponding penalty coefficient with actual value;kkFor the corresponding penalty coefficient of frequent discharge and recharge;Qt(st,at) it is t-1 times The optimal action value function formed after action;α is Studying factors;PsmaxFor the energy storage device active power upper limit;PsminSet for energy storage Have the work(lower limit of the power.
Embodiment
The invention will be further described below in conjunction with the accompanying drawings.
The present invention provides a kind of energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power, to solve Certainly wind stores up the control problem of energy storage device output power in hybrid system.
1. market environment leeward stores up hybrid system cooperative mechanism
Wind-powered electricity generation forms wind storage hybrid system with energy storage, wherein, wind-powered electricity generation is the energy source of hybrid system, and energy storage is then mixed Syzygy system provides the possibility of power adjusting control.Under Power Market, when hybrid system declares each to power grid in advance The generation schedule and spare capacity of section, in the process of running, hybrid system makes to mix by adjusting energy storage device output power in real time The total output power of syzygy system tracks generation schedule set in advance as far as possible.Due to the uncertainty of wind-powered electricity generation, actual wind-powered electricity generation is defeated Go out between power and plan power generation and deviation unavoidably occur, thus need power grid to provide spare electricity, mixed stocker for hybrid system System pays corresponding spare electricity expense, if in addition, deviation is excessive, the spare capacity bought beyond hybrid system, also needs Pay corresponding rejection penalty.To track generation schedule as far as possible, spare electricity expense and spare rejection penalty, mixed stocker are reduced System must make full use of the fireballing feature of energy storage device power adjusting, and energy-storage system charge-discharge electric power is controlled, will be mixed System power output bias control in the reasonable scope.
From the operating mechanism of wind storage hybrid system, hybrid system plan output power with declare spare capacity and determine Under conditions of, how making hybrid system real output, value performs according to schedule as far as possible, is to reduce total spare expense It is crucial.The fireballing feature of energy storage device power adjusting is made full use of, energy storage device is appropriately controlled, can be to a certain degree The upper fluctuation for reducing hybrid system output power, while reduction hybrid system spare expense, reduces its output power ripple Dynamic harmful effect of the property to power grid.
2. wind stores up hybrid system Controlling model
To realize maximally utilizing for wind-power electricity generation, air-blower control is controlled frequently with maximal power tracing, i.e., wind-powered electricity generation is actual Output power is determined do not have controllability completely by wind speed.The control of key, that is, energy storage device of wind storage hybrid system control, energy storage Equipment need to meet the requirement of related operation constraint, in charging, discharge process, need to meet that energy balance constrains, i.e.,:
In formula:Es,tFor t period initial time energy storage device storing electricities;Es,t+1Deposited for t period ends moment energy storage device Reserve of electricity;It is energy storage device in t period reality output active power, on the occasion of representing to send power, negative value represents to absorb work( Rate;ηcFor energy storage device charge efficiency;ηdFor energy storage device discharging efficiency;Δ t is the duration of t periods.
Energy storage device needs to meet output power constraint, capacity-constrained requirement in operation:
Esmin≤Es,t≤Esmax (3)
In formula:PsmaxFor the energy storage device active power upper limit;PsminFor energy storage device active power lower limit;EsmaxFor energy storage The place capacity upper limit;EsminFor energy storage device lower bound of capacity.
The charging and discharging state for defining energy storage device is Ks,t, Ks,tFor integer, represent that cut-off to t periods initial time ends, Energy storage device trickle charge or the when hop count of continuous discharge.If Ks,tFor positive number, then it represents that energy storage device continuous discharge Ks,t A period;If Ks,tFor negative, then it represents that energy storage device trickle charge | Ks,t| a period.When the energy storage device t+1 periods are initial The charging and discharging state K at quarters,t+1By Ks,tDischarge and recharge behavior with the t periods together decides on:
In formula:KsmaxFor energy storage device exempt from punishment when hop count, when energy storage trickle charge or electric discharge KsmaxAfter a period, this When change charging, discharge condition be not considered as frequent discharge and recharge.
Wind storage hybrid system control model is illustrated by taking the control process of certain period t as an example below.In hybrid system Before operation to the t periods, hybrid system is to the t period unscheduled power values that power grid is declaredSpare capacity isBy In the uncertainty of wind power output power, t period wind-powered electricity generation real outputsWith planned valueInevitably exist inclined Poor Δ Ph,t, i.e.,:
ΔPh,tAbsolute value is bigger, and spare expense is bigger required for hybrid system., can after by the way of wind storage is cooperated By controlling energy storage device charge-discharge electric power compensation system power output deviation, to make hybrid system that there is real-time monitoring ability, Reduce spare electricity expense and the spare rejection penalty that hybrid system is paid.In control strategy in the present invention, first according to dynamic The desired value of the definite hybrid system power deviation control of the strategy that electsHybrid system passes through to energy storage device output work The control of rate makes the general power deviation of hybrid system reach the desired value as far as possible, according toEnergy storage device can be tried to achieve Output power requirements
Due to being limited by energy storage device operation constraint, energy storage device real output valueIt may be unable to reach by formula (6) Obtained requirementsTherefore energy storage device real output is finally determined by feasibility verification:
IfEnergy storage device need to be verified and send powerAfterwards, if beyond energy storage device lower bound of capacity, if exceeding Then need to correct:
In formula:To consider the energy storage device output power correction value after capacity-constrained.
IfEnergy storage device absorbed power need to be verifiedAfterwards, if beyond energy storage device maximum size, if exceeding Then need to correct:
Energy storage device real output should also meet the constraint of the active power upper limit, lower limit constraint, if exceeding active power The upper limit or lower limit, then need to correct:
In formula:For energy storage device the t periods actual charge-discharge electric power.
In energy storage device real outputUnder the action of, t period hybrid system real output deviationsFor:
After determining t period system controlling behaviors, you can calculate hybrid system always spare expense C according to practical operation situationT (t):
CT(t)=Cr(t)+Cp(t) (11)
Cr(t) it is spare electricity expense, by the actual power deviation of t period hybrid systemsDetermine with spare electricity price It is fixed:
In formula:λr,tFor the spare electricity price of t periods.
Cp(t) it is spare beyond spare caused rejection penalty is declared to actually use, when the reality of t period hybrid systems Power deviation has exceeded the spare capacity R that system has been declaredh,t, then the part exceeded needs to receive punishment.Cp(t) can be by following formula Calculate:
In formula:λpFor hybrid system actual use it is spare beyond declare spare when corresponding penalty coefficient.
From formula (10), in the case where energy storage device output power, capacity are unrestricted, it can be achieved that reality output work( Rate strictly tracks Plan Curve, i.e.,Make total spare expense CT(t) it is 0.However, by economy, technical condition Restrict, energy storage device still falls within scarce resource, its operation reserve need to be optimized according to practical operation situation, determines appropriately Energy storage device output power, to realize the maximization of energy storage device effectiveness.Involved by the present invention it can be seen from above-mentioned Controlling model And the period relevance that the complex nature of the problem essentially consists in the uncertainty of wind power output power, energy storage Constraint is brought, this Outside, also need to consider energy storage device capacity limit, output power limit, frequent discharge and recharge limitation etc., the above problem is interweaved, and gives Wind storage the definite of hybrid system unscheduled power Tracking Control Strategy brings difficulty, and traditional optimization is difficult to effectively solve.
3. energy storage device power control strategy formulating method
3.1 intensified learnings theory and SARSA learning algorithm basic principles
Intensified learning is interacted by learning system and its local environment, constantly obtains environmental feedback letter in the process Breath is learnt, and with the passage of time and the accumulation of information, learning agent is persistently modified decision-making capability, progressively possesses To the decision-making capability of a certain problem.The learning process of intensified learning only needs itself to undergo, and contains probabilistic control to solution Problem has some superiority, for the wind storage hybrid system Optimal Control Problem under Uncertain environments according to the present invention also Well adapting to property.
SARSA study is a kind of unrelated nitrification enhancement of model, the return immediately of adoption status-action pair during iteration (s, a) is used as estimation function by value r and Q.Qπ(s, concrete meaning a) are represented by, and by state s, and are selected after acting a, The desired value of cumulative award is obtained under tactful π:
In formula:γ is discount factor.
The iterative process of SARSA study is generally carried out by formula (15):
Qt+1(st,at)=Qt(st,at)+α[rt+1(st,at)+γQt(st+1,at+1)-Qt(st,at)] (15)
In formula:stHybrid system state in which when being selected for the t times action;atFor the control taken during the t times action selection Braking is made;rt+1(st,at) for return value immediately, represent system in stUsing action a under statet, environmental feedback is to learning system Enhanced signal;Qt(st,at) acted for t-1 time after the optimal action value function that is formed;α is Studying factors.
In practical applications, the main problem of SARSA learning methods is " state-action " caused dimension excessive to quantity Calamity, since the requirement of SARSA learning methods is in the training process to all " state-action " to all traveling through enough numbers, Therefore, excessive " state-action " be not to by the generation of strong influence training effectiveness and available strategy, influencing control effect On the premise of, should reduce as far as possible " state-action " to quantity.In problem involved in the present invention, hybrid system is related to State variable include:Spare capacity that unscheduled power value, the hybrid system that hybrid system is declared are declared, energy storage device storage electricity Amount, energy storage device charging and discharging state, wind-powered electricity generation actual power etc.;The variable that controls of hybrid system active is energy storage device output work Rate.For reduce as far as possible " state-action " to quantity, this patent form state space and during motion space to above-mentioned variable into Row equivalence changes, on the premise of question essence and control effect is not influenced, as far as possible reduce " state-action " to number Amount, improves training effectiveness.
In state variable, " the unscheduled power value that hybrid system is declared ", " wind-powered electricity generation actual power " two states are all spread In a larger scope, if directly as state variable, it is carried out the number of states that is formed after discretization compared with Greatly.In fact, due to being to reduce hybrid system unscheduled power and actual power deviation the present invention relates to the key of problem, because Above-mentioned two state variable is converted into a deviation and remains to effective state for representing hybrid system by this, you can will " wind-powered electricity generation reality The deviation of border power and hybrid system unscheduled power " is used as a state variable, instead of " the unscheduled power that hybrid system is declared Value ", " wind-powered electricity generation actual power " two state variables, on the one hand, the number of state variable is reduced, on the other hand, due to " wind-powered electricity generation Actual power and the deviation of hybrid system unscheduled power " is departure, its distribution is compared with " the plan work(that hybrid system is declared Small more of rate value ", " wind-powered electricity generation actual power " two state variables, discretization, the state of generation are carried out using identical interval Number is largely eliminated.
In state variable, " energy storage device charging and discharging state " represents energy storage trickle charge, the number of continuous discharge, when it Sustained periods of time number reaches hop count when exempting from punishment and then continues to add up and control effect will not be had an impact, and only can increase the change Corresponding status number is measured, therefore, in " energy storage device charging and discharging state " variable update, if sustained periods of time number is more than punishment is exempted from Hop count, hop count when being set as exempting to punish by its value, the status number increase brought so as to avoid redundant state.
In motion space, since energy storage device output power is distributed in larger scope, if directly using " energy storage device Output power " is used as working value, it is necessary to sets up more working value, causes possible action collection larger.In this regard, this patent will be " mixed Syzygy system output power bias target value " is used as working value, which is departure, with " energy storage device output power " Compare, the power bracket of the departure is much smaller, and passes through known " hybrid system power planning value ", " wind-powered electricity generation actual power " Etc. state variable, can obtain rapidly corresponding " energy storage device output power ", from control effect for be equivalent, adopt Discretization is carried out with identical interval, set action number can greatly reduce.
During wind storage hybrid system control, due to using amount " hybrid system output power bias target indirectly Value " represents controlling behavior, if the controlling value sets unreasonable, is likely to occur " energy storage device output power " and does not reach the control Target processed, in this regard, this patent sets corresponding punishment, after training and study, ripe control plan in Reward Program immediately Summary can avoid such unreasonable control action automatically.
Since the frequent discharge and recharge of energy storage device can influence the energy storage device service life, need to consider during control to frequent The control of discharge and recharge behavior, in this regard, invention defines " energy storage device charging and discharging state ", sets according to energy storage in the training process Standby practical operation situation, the behavior to the frequent discharge and recharge of energy storage device are punished accordingly in Reward Program immediately, real The compromise of existing energy storage discharge and recharge conversion frequent degree and control effect.When the energy storage charge or discharge duration, which reaches, exempts from punishment Duan Hou, changing charging and discharging state will not pay for;Exempt to punish the period when the energy storage charge or discharge duration does not arrive, by root The behavior changed according to charging, electric discharge sustained periods of time number to its charging and discharging state is punished.
3.2 ambient condition collection
In the present invention, using the storing electricity E of period t initial time energy storage devices,t, wind-powered electricity generation actual power is with mixing The difference Δ P of system planning valueh,t, the spare capacity declared of hybrid systemEnergy storage device charging and discharging state Ks,tFour variables Represent hybrid system state.By the discrete form for turning to section of the first two variable, each section represents a state of the variable, Both take the form of centrifugal pump afterwards, and each centrifugal pump represents a state of variable, four cartesian product composition problem State set s.
By energy storage device capacity according to Δ Es=(Esmax-EsminThe length of)/m is divided into m section, and each section represents One state of stored energy capacitance:[Esmin,Esmin+ΔEs)、[Esmin+ΔEs,Esmin+2ΔEs)、…、[Esmax-ΔEs,Esmax]。
It is similar, by wind-powered electricity generation actual power and the difference Δ P of hybrid system planned valueh,tAccording toLength be divided into n area Between form:…、N is Even number.
The spare capacity that hybrid system is declaredAccording toLength be divided into j discrete state:
Energy storage device charging and discharging state shares k=2Ksmax+ 1 state:-Ksmax、-Ksmax+1、…、-1、0、1、…、 Ksmax-1、Ksmax
According to above-mentioned division, the ambient condition residing for hybrid system can be divided into m × n × j × k kind states:
S={ s1,s2,…,sm×n×j×k} (16)
State demarcation is thinner, to the more accurate of hybrid system operating status description;But meticulous state can cause state set Element number is excessive in conjunction, causes learning cycle long, is unfavorable for On-line Control, therefore in actual operation, need to be according to reality Need and empirically determined appropriate value.
3.2 possible action collection
In the present invention, by t period hybrid system output power deviation control targe valuesAs the feasible of control system Action, because the optional space presentation of hybrid system power offset value is symmetrical centered on 0, if adjacent control targe value Difference isHave for the output power deviation control targe value of setting:…、 0、…、Common 2b+1 possible action, uses a respectively1、a2、…、a2b+1Represent above-mentioned each target Value, composition possible action collection A:
A={ a1,a2,…,a2b+1} (17)
2.3 return immediately
In the present invention, the total spare expense, the feasibility of control targe, energy storage discharge and recharge for considering hybrid system are frequent Degree, the return value immediately for defining the hybrid system t periods are:
rt+1(st,at)=CT(t)+Ca(t)+Cc(t) (18)
CT(t) the spare expense that need to be paid for the hybrid system t periods, shown in its calculation formula such as formula (11-13).
Ca(t) punishment of set control targe value setting is not reached for t period hybrid system Actual Control Effect of Strong.By In the limitation of energy storage device capacity and power, if the hybrid system output power deviation control targe value set by control strategyIt is unreasonable, energy storage device may be caused to be unable to reach the control targe, show as energy storage device power demand values and reality Output valve differs, and to this unreasonable situation, is accordingly punished in return value immediately, calculation formula is:
In formula:kcCorresponding penalty coefficient is not inconsistent it with actual value for energy storage device power demand values.
Cc(t) corresponding punishment is changed for t period energy storage device charge and discharges state.When energy storage device trickle charge or continuously put The electric period reaches KsmaxAnd during the above, illustrate that energy storage changes charging and discharging state and is not belonging to frequent discharge and recharge at this time, without punishment;Instead It, if energy storage device trickle charge or continuous discharge and not up to KsmaxA period, illustrates that changing charging and discharging state at this time belongs to Frequent discharge and recharge, and | Ks,t| the smaller explanation charging and discharging state transformation of value is more frequent, and corresponding punishment is more.Cc(t) calculating is public Formula is:
In formula:kkFor the corresponding penalty coefficient of frequent discharge and recharge.
2.3.4 selection strategy is acted
, need to be according to current state s in learning processt, value function Qt(s, a) selects control action at.If action choosing every time Select all according to the maximum action of Q values selection, that is, perform greedy strategy, then possible action can not fully be explored, easily made Into local convergence;If the randomness of selection is too strong, pace of learning is too slow, is unfavorable for forming final control strategy.In this regard, this Invention is as follows using the action selection strategy of gradual change type:
1) random number T is taken in [0,1] sectionR
2) probability threshold values T is calculatedH
In formula:NnTo have been carried out acting number, the N of selectionTFor action selection total degree in default training.
3) action selection is carried out, if TR> TH, randomly choose any possible action from possible action collection equal probability and perform; If TR≤TH, possible action is selected according to greedy strategy, i.e.,:
According to selection strategy is acted, at the initial stage of study, action selection is tended to randomly choose, so as to possible action Fully explored, concomitant learning process, reduction acts the randomness of selection, is increasingly prone to greedy strategy, when action selects Number reaches 0.9NTAfterwards, system starts to perform greedy strategy, NTSetting according to hands-on effect and need given.
2.4 energy storage device power control strategies formulate step
The present invention is applied to the energy storage device power control strategy formulating method specific steps of wind-powered electricity generation unscheduled power tracking such as Under, flow chart is as shown in Figure 1:
1) basic data is inputted, including:(spare electricity expense, punishment take all kinds of expenses that hybrid system is merchandised with power grid With);Unscheduled power that hybrid system is declared, spare capacity, wind-powered electricity generation real output;Energy storage device power limit, capacity limit Value, efficiency for charge-discharge, exempt from hop count during punishment;SARSA learning algorithms parameter (Studying factors, discount factor, study step number, punishment Coefficient) etc.;
2) Q value function is initialized;
3) hop count t=1, executed action selection times N when settingn=1;
4) according to energy storage device storing electricity Es,t, wind-powered electricity generation actual power and hybrid system planned value difference Δ Ph,t, energy storage Charging and discharging state Ks,t, the spare capacity declared to power grid of hybrid systemDetermine present period state in which st
5) random number T is generatedR, probability threshold values T is calculated according to formula (21)HIf TR≤TH, greedy plan is selected according to formula (22) Slightly acted for this periodIf TR> TH, selection acts a from possible action collection A at randomt
6) according to action atWithOne-to-one relationship obtain hybrid system output power deviation control targe value;
7) energy storage device demand power is calculated according to formula (6)The actual discharge and recharge of energy storage device is calculated according to formula (7-9) Power
8) t period hybrid system real output deviations are calculated according to formula (10)
9) the return value r immediately of t periods is calculated according to formula (10-13), formula (18-20)t+1(st,at);
10) energy storage charging and discharging state is corrected according to formula (4);
11) the state s of t+1 periods is identifiedt+1, determine that the t+1 periods act a according to action selection strategyt+1
12) Q value function is modified according to formula (15);
If 13) study step number as defined in reaching, training terminate, it is what is formed to perform greedy strategy according to Q value function Hybrid system control strategy;If study step number, t=t+1, N as defined in not up ton=Nn+ 1, go to step 4).

Claims (3)

1. a kind of energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power, it is characterized in that, the method exists It is theoretical based on intensified learning under electricity market background, wind storage hybrid system output power is formulated using SARSA learning methods Strategy, comprises the following steps that:
A. basic data is inputted, including:Hybrid system and all kinds of cost parameters of power grid transaction;The plan work(that hybrid system is declared Spare capacity that rate, hybrid system are declared, wind-powered electricity generation real output;Energy storage device power limit, capacity limit value, discharge and recharge effect Rate, exempt from hop count during punishment;SARSA learning algorithm parameters;Wherein, hybrid system includes standby with all kinds of cost parameters that power grid is merchandised Power consumption price, spare penalty coefficient;SARSA learning algorithms parameter includes Studying factors, discount factor, study step number and punishment Coefficient;
B. Q value function is initialized;
The action selection times N that hop count t=1, setting have been carried out when c. settingn=0;
D. according to energy storage device t period initial time storing electricities Es,t, wind-powered electricity generation actual power and wind storage hybrid system planned value it Poor Δ Ph,t, energy storage device charging and discharging state Ks,t, the spare capacity declared of hybrid systemDetermine that t periods wind stores up mixed stocker State in which of uniting st
E. according to current state st, value function Qt(s, a) selects control action at, comprise the following steps that;
1. take random number T in [0,1] sectionR
2. calculate probability threshold values TH
In formula:NnTo have been carried out acting the number of selection;NTThe total degree of selection is acted for training overall process;
3. carry out action selection:If TR> TH, selection acts a from possible action collection A at randomt;If TR≤TH, select greedy strategy Acted as this period, i.e.,:
F. according to action atWith hybrid system power deviation control targe valueOne-to-one relationship, obtaining the t periods mixes The desired value of system output power deviation control;
G. according to hybrid system output power deviation control targe valueCalculate energy storage device Power Control requirements
H. actual charge-discharge electric power of the energy storage device in the t periods is calculated
If the control setting value of t period energy storage devicesEnergy storage device need to be verified and send powerAfterwards, if beyond energy storage Place capacity lower limit, needs to correct if if:
In formula:Es,tFor t period initial time energy storage device storing electricities;ηdFor the discharging efficiency of energy storage device;Δ t holds for the period The continuous time;EsminFor energy storage device lower bound of capacity;For the nonce kept in when asking for energy storage real output;
IfEnergy storage device absorbed power need to be verifiedAfterwards, if beyond energy storage device maximum size, needed if if Correct:
In formula:ηcFor the charge efficiency of energy storage device;EsmaxFor energy storage device maximum size;
In addition, energy storage device real output should also meet the constraint of the active power upper limit, lower limit constraint, if exceeding active power The upper limit or lower limit, then need to correct:
In formula:PsmaxFor the energy storage device active power upper limit;PsminFor energy storage device active power lower limit;It is real for energy storage device Border output power;
I. t period hybrid system real output deviations are calculated
In formula:For t period wind-powered electricity generation real outputs;For hybrid system t period unscheduled power values are declared to power grid;
J. the return value r immediately of t periods is calculatedt+1(st,at):
rt+1(st,at)=Cr(t)+Cp(t)+Ca(t)+Cc(t)
Cr(t) it is spare electricity expense, Cp(t) it is spare beyond spare caused rejection penalty is declared to actually use, calculate public Formula is respectively:
In formula:λr,tFor the spare electricity price of t periods;λpFor hybrid system actual use it is spare beyond declare redundancy window when, Rejection penalty coefficient of the power grid to hybrid system;Rh,tThe t period spare capacities declared for hybrid system;
Ca(t) rejection penalty of set control targe value setting, meter are not reached for t period hybrid system Actual Control Effect of Strong Calculating formula is:
In formula:kcCorresponding penalty coefficient is not inconsistent it with actual value for energy storage device power demand values;
Cc(t) corresponding punishment is changed for t period energy storage device charge and discharges state:
In formula:kkFor the corresponding penalty coefficient of frequent discharge and recharge;Ks,tFor the continuous charge and discharge sustained periods of time number of energy storage, positive number represents Hop count during trickle charge, hop count during negative number representation continuous discharge;KsmaxHop count during to exempt from punishment, when energy storage trickle charge or Discharge KsmaxAfter a period, change charging, discharge condition will not pay for;
K. according to Ks,tWith the discharge and recharge behavior renewal energy storage device t+1 period charging and discharging states K of t periodss,t+1
L. t+1 periods wind storage hybrid system state in which s is identifiedt+1, determine that the storage of t+1 periods wind is mixed according to action selection strategy Syzygy system control action at+1
M. Q value function is modified:
Qt+1(st,at)=Qt(st,at)+α[rt+1(st,at)+γQt(st+1,at+1)-Qt(st,at)]
In formula:Qt(st,at) it is by t-1 revised optimal action value function;α is Studying factors;γ is discount factor;
If study step number N as defined in n. reachingT, training terminates, according to Q value function execution greedy strategy, that is, control strategy;If do not reach To defined study step number, t=t+1, Nn=Nn+ 1, go to step d.
2. a kind of energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power according to claim 1, It is characterized in that the wind storage hybrid system state in which stIt is the element in wind storage hybrid system state set S, the wind stores up The definite method of hybrid system state set S is as follows:
By energy storage device capacity according toSiding-to-siding block length be divided into m section:[Esmin,Esmin+Δ Es)、[Esmin+ΔEs,Esmin+2ΔEs)、…、[Esmax-ΔEs,Esmax];By wind-powered electricity generation actual power and hybrid system planned value it Poor Δ Ph,tAccording toSiding-to-siding block length be divided into n section:N is even number, will The spare capacity that hybrid system is declaredAccording toLength be divided into j discrete state:0、 In addition, energy storage device charging and discharging state shares k=2Ksmax+ 1 state:-Ksmax、- Ksmax+1、…、-1、0、1、…、Ksmax-1、Ksmax;Wind storage hybrid system shape is formed by the cartesian product of aforementioned four variable states State set S:
S={ s1,s2,…,sm×n×j×k}。
A kind of 3. energy storage device control strategy formulation side for the tracking of wind-powered electricity generation unscheduled power according to claim 1 or 2 Method, it is characterized in that, the building method of the possible action collection A is as follows:
By t period hybrid system output power deviation control targe valuesAs the action behavior of control system, if adjacent control The difference for setting up definite value isThe setting value is controlled to be Common 2b+1 possible action, uses a respectively1、a2、…、a2b+1Generation The above-mentioned each control targe value of table, composition possible action collection A:
A={ a1,a2,…,a2b+1}
In formula:a1,a2,…,a2b+1Above-mentioned 2b+1 control setting value is represented respectively.
CN201710470529.3A 2017-06-20 2017-06-20 Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power Expired - Fee Related CN107026462B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710470529.3A CN107026462B (en) 2017-06-20 2017-06-20 Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710470529.3A CN107026462B (en) 2017-06-20 2017-06-20 Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power

Publications (2)

Publication Number Publication Date
CN107026462A CN107026462A (en) 2017-08-08
CN107026462B true CN107026462B (en) 2018-05-04

Family

ID=59531467

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710470529.3A Expired - Fee Related CN107026462B (en) 2017-06-20 2017-06-20 Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power

Country Status (1)

Country Link
CN (1) CN107026462B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108090666B (en) * 2017-12-13 2020-07-10 华中科技大学 AA-CAES-containing power grid electric energy and reserve capacity cooperative scheduling method
CN110414725B (en) * 2019-07-11 2021-02-19 山东大学 Wind power plant energy storage system scheduling method and device integrating prediction and decision
CN113988897A (en) * 2021-09-14 2022-01-28 广西电网有限责任公司 Wind storage system output deviation punishment cost calculation method and device
CN114017904B (en) * 2021-11-04 2023-01-20 广东电网有限责任公司 Operation control method and device for building HVAC system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102522776B (en) * 2011-12-23 2014-01-29 中国电力科学研究院 Method for improving wind power tracking capability on planned output by energy storage system
CN104779631B (en) * 2014-12-31 2017-07-18 国家电网公司 Energy storage tracking wind power output method of planning and its system based on the pre- power scale of wind-powered electricity generation
CN104701873B (en) * 2015-03-27 2018-12-11 国家电网公司 A kind of battery energy storage system optimal control method contributed for tracking wind-powered electricity generation plan
CN106684913B (en) * 2016-12-29 2023-05-23 中国电力科学研究院 Multi-agent-based energy storage power station tracking power generation plan control system and method

Also Published As

Publication number Publication date
CN107026462A (en) 2017-08-08

Similar Documents

Publication Publication Date Title
CN107026462B (en) Energy storage device control strategy formulating method for the tracking of wind-powered electricity generation unscheduled power
CN109962499B (en) Power grid multi-time scale scheduling method
CN107800157B (en) Double-layer optimized scheduling method for virtual power plant containing polymer temperature control load and new energy
CN103683337B (en) A kind of interconnected network CPS instruction dynamic assignment optimization method
CN107248751A (en) A kind of energy storage station dispatch control method for realizing distribution network load power peak load shifting
WO2017000853A1 (en) Active power distribution network multi-time scale coordinated optimization scheduling method and storage medium
CN103971181B (en) A kind of economic load dispatching method a few days ago for virtual plant
CN108667052A (en) A kind of polymorphic type energy-storage system planning and configuration method and system of Virtual optimal power plant operation
CN108110795B (en) Wind power plant group outgoing power transmission capacity and energy storage configuration joint optimization method
CN106548291A (en) A kind of micro-capacitance sensor on-road efficiency distribution method based on Shapley values
CN104616069A (en) Annual power generation plan rolled decomposition optimization method taking balance between plan finishing rate and load rate into consideration
CN116345577B (en) Wind-light-storage micro-grid energy regulation and optimization method, device and storage medium
CN107392420A (en) A kind of household energy management system intelligent control method based on demand response
CN109872228A (en) Meter and probabilistic flexible load polymerization quotient participate in electricity market Bidding system
CN107706932A (en) A kind of energy method for optimizing scheduling based on dynamic self-adapting fuzzy logic controller
Yu et al. Energy storage arbitrage in grid-connected micro-grids under real-time market price uncertainty: A double-q learning approach
CN112529249B (en) Virtual power plant optimal scheduling and transaction management method considering green certificate transaction
CN116720690A (en) Multi-virtual power plant electricity-carbon joint optimization operation method, equipment and storage medium
Wu et al. Risk-constrained day-ahead scheduling for gravity energy storage system and wind turbine based on IGDT
CN107147115A (en) A kind of user side energy management method based on multiple time scale model
Yang et al. Data-driven optimal dynamic dispatch for Hydro-PV-PHS integrated power systems using deep reinforcement learning approach
Mandić et al. A general model of optimal energy storage operation in the market conditions
Ghavidel et al. Hybrid power plant bidding strategy including a commercial compressed air energy storage aggregator and a wind power producer
CN116050637A (en) Comprehensive energy virtual power plant optimal scheduling method and system based on time-of-use electricity price
Zhang et al. Transfer deep reinforcement learning-based large-scale V2G continuous charging coordination with renewable energy sources

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20180504

Termination date: 20190620

CF01 Termination of patent right due to non-payment of annual fee