AU2021106780A4

AU2021106780A4 - Virtual power plant self-optimisation load track control method

Info

Publication number: AU2021106780A4
Application number: AU2021106780A
Authority: AU
Inventors: Liangyu Chen; Guangyu He; Zhiyong Li; Chaorong MO; Tianhong REN; Jie Shao; Qing Wu; Xiaowei Yan; Huan ZHOU
Original assignee: Hainan Electric Power School Hainan Electric Power Technical School; SHANGHAI QIANGUAN ENERGY SAVING TECHNOLOGY CO Ltd; Shanghai Jiaotong University
Current assignee: Hainan Electric Power School Hainan Electric Power Technical School; Shanghai Qianguan Energy Saving Technology Co Ltd; Shanghai Jiaotong University
Priority date: 2020-11-17
Filing date: 2021-08-24
Publication date: 2021-11-18
Anticipated expiration: 2029-08-24
Also published as: CN112448396B; CN112448396A

Abstract

OF THE DISCLOSURE The present disclosure discloses a virtual power plant self-optimization load track control method in the technical field of electrical engineering and automation thereof. The method comprises the following steps: setting an evaluation index of self-optimization track control of a virtual power plant; establishing an event-driven stimulation-feedback control mechanism, and realizing a DDQN-based training and decision-making method. The present disclosure, based on the self-optimization operation theory, drives the virtual power plant as a whole to track and operate an expected load target curve through massive heterogeneous and time-varying distributed resource self-adaptive adjustment in a bottom-to-top mode, and realizes dynamic mining and efficient utilization of the flexibility of the massive distributed resources; the method can rapidly ascertain the regulation and control capability of the virtual power plant in complex conditions, accurately track a target load curve within the capability range, and can effectively drive the whole to be close to a theoretical optimal operation point during the individual optimal operation. - 3/3 - - - - - - - - Z - - - - - - - I F - - - I~ II -------- a- -- - - - (D I -G I~ -i~ta

Description

- 3/3

- - Z - - - - - - - I F - - - - - - - - -

I~ II

-- - - - -------- a-

(D

I -G I~ -i~ta

VIRTURAL POWER PLANT SELF-OPTIMIZATION LOAD TRACK CONTROL METHOD TECHNICAL FIELD

[01] The present disclosure relates to the technical field of electrical engineering and automation thereof, and particularly, to a virtual power plant self-optimization load track control method.

BACKGROUNDART

[02] A virtual power plant can aggregate distributed power, energy storage, and various types of loads (residential, commercial, and industrial) scattered in various areas according to certain rules. It is essentially a typical complex system, as shown in FIG. 1; for such a complex system composed of massive amounts of heterogeneous distributed resources, it is difficult to solve such complex problems with purely mathematical methods. Meanwhile, the existing control methods mainly have the following shortcomings: (1) most of the traditional control methods (such as multi-regional virtual power plant integrated energy coordination and scheduling optimization model [J]. Proceedings of the Chinese Society of Electrical Engineering, 2017, 23(37): 27-37+316; Rahimiyan M, Baringo L. Real-time energy management of a smart virtual power plant

[J]. IET Generation transmission & Distribution, 2018, 13(11): 2015-2023, and Abdolrasol M G M, Hannan, MA, MohamedA, et al., An Optimal Scheduling controller for Virtual Power Plant and Microgrid Integration using Binary Backtracking Search Algorithm [J]. IEEE Transactions on Industry Applications, 2018:1-1) face hundreds or thousands of objects, and grid-friendly objects, such as conventional units and energy storage systems, are the main ones, and they have fewer uncertainties. In the future, traditional methods will face a large number of renewable energy with different goals, scattered property rights, and time-varying status, which will bring huge challenges to the top-down approaches. (2) Some decentralized control methods (such as Shuai F, Jiang L, Qing W, et al. Optimal coordination of virtual power plant with photovoltaics and electric vehicles: Atemporally coupled distributed online algorithm, Applied Energy, 2020; Xue DONG, Shuqin TU, Ye LI, et al., Dynamic Pricing and Energy Management of Multi-Virtual Power Plants Based on Master-Slave Game based on Meta-Model Optimization Algorithm [J]. Power System Technology, 2020, 44(03): 973-983 and Mehran Jafari, Asghar Akbari Foroud. A medium/long-term auction-based coalition-forming model for a virtual power plant based on stochastic programming [J]. International Journal of Electrical Power&Energy Systems, 2020, 118) use a decentralized strategy to effectively enhance the control performance of virtual power plant; however, there are still many flaws in the actual application process; for example, the control method is still adopting a top-down approach, and distributed resources are relatively passive during the operation, do not have autonomy, and have poor environmental adaptability. There is strong coupling between each other, which makes the online expansion capability of the system weak. Moreover, suppose the participants are completely rational and the decision-making process relies on complete information, etc. These flaws make the optimization results differ greatly from practical application.

"Self-optimization" is a theoretical system established by the team of Academician LU Qiang of Tsinghua University when researching and implementing smart grids. It is mainly based on the electric hybrid control theory to make an object realize the transition from an unsatisfactory state to a satisfactory state through its own automatic adjustment. In order for the virtual power plant to have self-optimization operation capability, three main problems need to be solved:

[03] 1) what is optimal

[04] A set of multiple indicators refers to a set of indicators that the virtual power plant needs to achieve in the process of responding to external regulation instructions and coordinating the operation of internal participants, and each indicator in the indicator set can be quantified in real time through measurement and calculation.

[05] How to be optimal

[06] An event-driven operating mechanism, based on power hybrid cybernetics, defines a satisfactory boundary for each indicator in the indicator set. If the value of the indicator running in real time crosses the boundary, it will be in an unsatisfactory state. The transition of any indicator from a satisfactory state to an unsatisfactory state is classified and hierarchically defined as an event. The occurrence of an event will trigger the corresponding procedures to coordinate and control until the event is eliminated, and all indicators return to a satisfactory state, then the overall operating status of the system is considered satisfactory and continuously improving. This method can ensure that the operating effect of the virtual power plant is sufficiently satisfactory in the course of engineering practice.

[07] 3) how to be self-optimal

[08] A bottom-up realization method, including: 0 situational awareness technology from the individual to the whole; among them, individuals can recognize the identity, status, and events of the terminal based on historical data and real-time measurement values, thereby indirectly determining the user's behavior. The overall level can judge whether the current state is satisfactory based on the measured value of the bottom layer and according to the topology and the trend of the power grid. © unified standard modeling and interface. Participants can seamlessly integrate with the system by standardizing interactive data and behaviors. 0 automatic, spontaneous and adaptive control algorithm. System control realizes the transition from "people tell things" to "things tell people", and individuals can continuously iterate and improve their own behaviors through interaction with the environment, and their behaviors are compatible with the overall goal. © strict and flexible rules. Individuals collaborate and compete under strict rule constraints, and establish dynamic logical coupling through self-organization. Rules can be adjusted appropriately according to changes in the environment, so as to achieve dynamic guidance to the behavior of participants. 0 Loosely coupled system architecture. Each participant is equal and independent, and is fully decoupled. The designed architecture can support online maintenance, fault tolerance and continuous growth of the system. In order to monitor the load of the virtual power plant, it is urgent to design a virtual power plant load track control method based on the self-optimizing operation theory. The invention designs a virtual power plant load track control method based on self-optimization operation theory; based on this, the present disclosure designs a virtual power plant self-optimization load track control method to solve the above problems.

SUMMARY

[09] The objective of the present disclosure is to provide a virtual power plant self-optimization load track control method to solve the problems proposed in the above background technology.

[10] In order to realize the above objective, the present disclosure provides the following technical solutions: virtual power plant self-optimization load track control method, comprising the following steps:

[11] Si: setting an evaluation index of self-optimization track control of a virtual power plant;

[12] S2: establishing an event-driven stimulation-feedback control mechanism;

[13] S3: realizing a DDQN-based training and decision-making method.

[14] Further, the step S Iis specially: in the load track control process, the virtual power plant coordinates distributed resources to increase or reduce load to achieve online matching of an expected target; since the virtual power plant provides external flexible services, it will obtain certain profits from external sources, such as power grids, power markets, and new energy operators, and meanwhile it needs to give a certain amount of compensation to the distributed resources that contribute internally; as for internal and external needs, defining following two indicators:

[15] Indicator 1: load track error (' AP t= ( t) t

[16] |it ={11,2,---, T}

[17] Wherein: P() and Ptt(t)are actual load and target load of the virtual power

plant at time period of t, is a time error of the time period; when AL()0, the

virtual power plant expects to reduce the load; otherwise, if ALO(t)<0, the virtual power plant expects to increase the load; T is a number of time periods of the research cycle;

[18] Indicator 2: subsidy cost

F(A)= Y'A,(t)APL(t

[19] (t)=2(t)/AP(t) (2)

[20] Intheformula: A(t) is a subsidy amount released by the virtual power plant at the time period, with a unit of (RMB) Yuan; the final subsidy uses a linear allocation manner, i.e., the profits obtained by any unit load change is an incentive coefficient85;

[21] For calculation of the indicators, a logical electricity meters in the system assumption is used for aggregate statistics and settlement; constraints that need to be considered at the virtual power plant level including upper and lower constraints of the track ability, a climbing rate, and the network constraints in the distribution network environment.

[22] Further, in step S2, in the load track control scenario, the events generated by the virtual power plant mainly include the following two: 1) track events, TE; 2) coordinated events, CE.

[23] Further, the establishing an event-driven stimulation-feedback control mechanism mainly includes the steps of:

[24] Step 1: a load bias triggers the TE; defining that a bias between the load at some period and the target load satisfy formula (3), trigger the TE; otherwise, remove the TE;

FAP (t)d>9 Pgt) P(gt)t)>Qh

[25] |APL(t)>Ec P, (t) Qth (3)

[26] In the formula,d is proportion coefficient,and isanerrorconstant.

[27] Step 2: a virtual power plant cloud service center, i.e., VPP-CSC, releases an initial stimulus signal; in order to guide the distributed resources to adjust to the established direction, after the TE is triggered, the VPP-CSC will release a four-tuple I, (t) stimulus signal '" , e.g.:

[28] I'-(t)=r[S,8(t),tS,tE] (4)

[29] In the formula: S is task code;

[30] According to assumption, in the initial state, the VPP-CSC is unable to accurately know a number and state of the online distributed resources and actions taken

possibly; thus, an initial incentive value (n)is obtained based on the historical load data; {JPU (j ,Pup (tP(-, L ini

[311 |j= (5)

[32] In the formula: ""w is a historical load of the virtual power plant in the past jthperiod; d is a referenced number of historical data;

[33] Step 3: the distributed resource feeds back adjustment plans according to stimulus signals, makes decision independently after receiving the stimulus signals, and feeds back the adjustment plans within the task period, i.e., load increasing (reducing) timing sequence:

[34] PDER,i(Sc)= [DER,i(t,''',PDER,,i(t] (6)

[35] In the formula, DER () is the load increasing (reducing) amount of the in distributed resource at the tthperiod;

[36] Step 4: the VPP-CSC determines whether to trigger the CE; before an expiration timestamp, counting a cumulative response amount fed back by the distributed resources, and triggering the CE if the cumulative amount satisfies formula (7): ' N(t) P1,(t)= PDER W

P,(t)-APL(t) PtM th

P7(t)- AP(t)> tt)>(7) 1371 L (7)

[38] In the formula: N(t) is a number of online distributed resources at period t,

P(o is a cumulative response time of the distributed resource feedback at the period;

[39] Step 5: iterating until the CE is removed:

[40] 5h (t) = (t)+oVD1(N(t)) (8)

[41] In the formula, 5 h(t) is an incentive coefficient generated at the hth iteration at

period t, cT is a predefined gradient step; Dh-'(N(t))is a deviation distance after h-I

times of iteration when the terminal number is N(t), until (t) removes the CE, the

distributed resource will add the coordination result into the task list again, and the CE removing condition is formulas (9)-(11):

[42] in G(t) . (9)

FP,(t) = AP(t) - (t)P,(t)=g(on ,t) 143] ~Pl(t)= APL(t)-c (10) 17,(t) = AP,t) + 1(t) P,(t)=g(8.ma,t)

[44] PL7 t ,(1 dt

[45] In the formula: g(,t)is a total response amount of the distributed resource

feedback when the incentive coefficient is 5 at period t, 5min and 'max are the theoretical minimum and maximum incentive coefficients to remove the error;

[46] Step 6: removing the TE; the distributed resource is self-regulated according to the task list, and the VPP-CSC determines whether the system satisfies the condition for removing the TE according to the check result, and performs a final calculation according to the verification result afterwards, the condition for removing the TE is:

FJp '(t)_I"(t)'P (t) P't:m

[48] In the formula, is an actual track load of the integral virtual power plant after the distributed resource self-regulation is finished.

[49] Further, the step S3 is specially: using the DDQN algorithm to adaptively optimize the decision-making process of the distributed resources, the training method is:

[50] For the feedback training of the distributed resources, since the load track process of the virtual power plant has nothing to do with the online quantity, status and characteristics of the distributed resources, for any load increasing or decreasing amount APL(t) , the virtual power plant will adjust the incentive coefficient 6(t) to realize

accurate track of targets and optimization of control costs, and according to the principle of linear profit distribution, the distributed resources are essentially based on the income of unit load adjustments to make decisions; thus, the purpose of training is to build a network, establishing a standard mapping connection between stimulus-state-behavior based on the action value function; the data set of the training process includes:

[51] Al) time-of-use electricity price data set

[521 =[C', C., C" (13)

[53] In the formula, v,f, and p denote peak, normal, and valley periods respectively,

C,, Ci , and C"o are time-of-use electricity prices at the peak, normal, and valley periods respectively;

[54] 2) state set

[55] SDER, =[sDERI,is' , sEER DER,,,' DER,; 4)

[56] In the formula, sDER,iis the xth state of the ith distributed resource, and4 is a number of states of the resource;

[57] 3) incentive coefficient data set

[58] CU ,5' ~( ] (15)

[59] In the formula, 'min andmax are the minimal value and the maximum value of

the adjusted profits, wherein the minimum value is greater than 0, i.e., 5min >0, the

maximum value is smaller than ",times of the current time-of-use electricity price, i.e.,

'5iax<20Ct,,,(t)

[60] Further, the DDQN algorithm mainly includes an evaluation network and a target network; during the off-line training, on one hand, action samples are recorded and stored in an experience pool D; on the other hand, by updating network parameters, the distributed resources can obtain optimal feedback behaviors by calculating the incentive coefficient; at the off-line phase, the parameters of the evaluation network are completely the same as those of the target network; after the distributed resources run online, the two networks will run asynchronously, wherein the evaluation network runs online to constantly update the network parameters and calculate an optimal behavior at; the target network evaluates Q value of the behavior, and the evaluation network

copies the parameter value to the target network once in a while.

[61] Compared with the prior art, the advantageous effects of the present disclosure are: the present disclosure, based on the theory of self-optimization operation, adopts a bottom-up approach, and drives the virtual power plant to track the expected load target curve as a whole through the adaptive adjustment of massive heterogeneous and time-varying distributed resources, thus realizing dynamic mining and efficient utilization of the massive distributed resource flexibility. The advantages are as follows: (1) Based on the theory of self-optimizing operation, a bottom-up operation method of virtual power plant is proposed, which can effectively deal with the challenge brought by the dynamic access of massive complex heterogeneous distributed resources to the coordinated control of the virtual power plants; different from being "optimized" and waiting for control commands, distributed resources are more active, and can make independent decisions and adaptive adjustments under the guidance of certain rules, so as to realize the optimal operation that an individual drives the whole;

[62] (2) Proposing an event-driven stimulus-feedback control mechanism. This mechanism can solve the problem that the online expansion capability of the existing coordination mechanism is not insufficient, decouples the optimization of the system from the number, type, and status of participants, and ensures the optimization operation of the system from an unsatisfactory state to a satisfactory state;

[63] (3) Establishing a set of bottom layer adaptive response rule; a set of general distributed resource response rules are established by the MDP model and action value function. The basis of the DDQN algorithm is to perform MDP modeling first, which eliminates the influence of uncertainty of independent decision-making of distributed resources on system performance, so that the individual decision-making is consistent with the overall goal; for the proposed DDQN optimization algorithm, on the one hand, the decision of the distributed resources no longer needs to depend on complete information; on the other hand, it is helpful for the distributed resources to adapt to the dynamic environment, so as to promote the virtual power plant to participate in the diversified application scenarios of the power grid; meanwhile, based on self-optimized operation theory, it realizes the automatic track control of the target load curve and the efficient utilization of massive flexible resources; the method can quickly ascertain the regulation and control ability of the virtual power plant under complex conditions, accurately track the target load curve within its ability range, and can effectively drives the whole be close to the theoretical optimal operating point while the individual tends to operate optimally.

BRIEFT DESCRIPTION OF THE DRAWINGS

[64] In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[65] FIG. 1 is a schematic view of an existing virtual power plant self-optimization load track control;

[66] FIG. 2 is a flow chart of the virtual power plant self-optimization load track control of the present disclosure;

[67] FIG. 3 is a schematic view of an end-to-end mapping structure of the distributed resource of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

[68] The following will describe technical solutions in the embodiments of the present disclosure clearly and completely in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall in the scope of protection of the present disclosure.

[69] In the description of the present disclosure, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inside", "outside" indicate orientation or positional relationship, which is based on the orientation or positional relationship illustrated in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, being constructed according to the specific azimuth, and cannot be understood as a limitation of the present disclosure.

[70] Refer to FIG. 2 and FIG. 3, the present disclosure provides a technical solution: A virtual power plant self-optimization load track control method, which comprises the following steps:

[71] Si: setting an evaluation index of self-optimization track control of a virtual power plant;

[72] S2: establishing an event-driven stimulation-feedback control mechanism;

[73] S3: realizing a DDQN-based training and decision-making method.

[74] Wherein step Si is specially: in the load track control process, the virtual power plant coordinates distributed resources to increase or reduce load to achieve online matching of an expected target; since the virtual power plant provides external flexible services, it will obtain certain profits from external sources, such as power grids, power markets, and new energy operators, and meanwhile it needs to give a certain amount of compensation to the distributed resources that contribute internally; as for internal and external needs, defining following two indicators:

[75] Indicator 1: load track error (, AP t= t t

[76] 1,2-,T} (1t

[77] Wherein: P() and Ptt(t)are actual load and target load of the virtual power

plant at time period of t, is a time error of the time period; when AP()0, the

virtual power plant expects to reduce the load; otherwise, if APL(t)<0, the virtual power plant expects to increase the load; T is a number of time periods of the research cycle;

[78] Indicator 2: subsidy cost

F(A)= Y'A, (t)APL(t

[791 (t)/AP (t) (2)

[80] Wherein: A(t) is a subsidy amount released by the virtual power plant at the time period, with a unit of (RMB) Yuan; the final subsidy uses a linear allocation manner, i.e., the profits obtained by any unit load change is an incentive coefficient85;

[81] For calculation of the indicators, using a logical electricity meters in the system assumption for aggregate statistics and settlement; constraints that need to be considered at the virtual power plant level including upper and lower constraints of the track ability, a climbing rate, and the network constraints in the distribution network environment.

[82] In step S2, in the load track control scenario, the events generated by the virtual power plant mainly include the following two: 1) track events, TE; 2) coordinated events, CE.

[83] Designing an event-driven stimulation-feedback control mechanism mainly includes the steps of:

[84] Step 1: a load bias triggers the TE; defining that a bias between the load at some period and the target load satisfy formula (3), trigger the TE; otherwise, remove the TE;

FAPL Pgt(t) d(t)>9 Igt(t)> th

[85] |AP(t)>s Pt(t) Mh (3)

[86] Intheformula,d is proportion coefficient,and isanerrorconstant;

[87] Step 2: a virtual power plant cloud service center, i.e., VPP-CSC, releases an initial stimulus signal; in order to guide the distributed resources to adjust to the established direction, after the TE is triggered, the VPP-CSC will release a four-tuple

I, (t)

stimulus signal '" , e.g.:

[88] I- (t)=r[S,8(t),tS,tE] (4)

[89] In the formula: S is task code;

[90] According to assumption, in the initial state, the VPP-CSC is unable to accurately know a number and state of the online distributed resources and actions taken

possibly; thus, an initial incentive value (t) is obtained based on the historical load data;

[911 |j= (5)

[92] In the formula: is a historical load of the virtual power plant in the past jthperiod; d is a referenced number of historical data;

[93] Step 3: the distributed resource feeds back adjustment plans according to stimulus signals, makes decision independently after receiving the stimulus signals, and feeds back the adjustment plans within the task period, i.e., load increasing (reducing) timing sequence:

[94] PDER =[s' ER, 1,'''),DR,i(t) (6)

t

[95] In the formula, PDER,) is the load increasing (reducing) amount of the i distributed resource at the tthperiod;

[96] Step 4: the VPP-CSC determines whether to trigger the CE; before an expiration timestamp, counting a cumulative response amount fed back by the distributed resources, and triggering the CE if the cumulative amount satisfies formula (7): N(t) P,(t)= PDER,i

P,(t)- AP (t) >Qet0') gt(0:Mth

P9(t- AP(t)>&E (0>th 1971 (7)

[98] In the formula, N(t) is a number of online distributed resources at period t,

is a cumulative response time of the distributed resource feedback at the period;

[99] Step 5: iterating until the CE is removed:

1

[100] 5,h _ h-l(t)+o=5D- (N(t)) (8)

[101] In the formula, ' ) is an incentive coefficient generated at the hth iteration at period t, 'T is a predefined gradient step; D,'-(N(t))is a deviation distance after h-i times of iteration when the terminal number is N(t), until g(') removes the CE, the distributed resource will add the coordination result into the task list again, and the CE removing condition is formulas (9)-(11):

[102] in3 g() W . (9)

F1,(t) = AP (t) - t P,(t)=g(on,t)

F1031P (t) = APt( ~t) P,(t)=g(8ma(,t)

[104] dP t '(1 L'7

[105] In the formula: g(,t)is a total response amount of the distributed resource

feedback when the incentive coefficient is 1 at period t, 5min and 'max are the theoretical minimum and maximum incentive coefficients to remove the error;

[106] Step 6: removing the TE; the distributed resource is self-regulated according to the task list, and the VPP-CSC determines whether the system satisfies the condition for removing the TE according to the check result, and performs a final calculation according to the verification result afterwards, the condition for removing the TE is:

[1071] '"'(t) - ,(t)| pd (t) >Q (12)

[108] In the formula, "(t) is an actual track load of the integral virtual power plant after the distributed resource self-regulation is finished.

[109] Step S3 is specially: using the DDQN algorithm to adaptively optimize the decision-making process of the distributed resources, the training method being:

[110] for the feedback training of the distributed resources, since the load track process of the virtual power plant has nothing to do with the online quantity, status and characteristics of the distributed resources, for any load increasing or decreasing amount APL(t), the virtual power plant will adjust the incentive coefficient 6(t) to realize

[111] 1) time-of-use electricity price data set

[1121 , =[C., ,C" (13)

[113] In the formula, v,f, and p denote peak, normal, and valley periods respectively,

C to Ci,, and C"o are time-of-use electricity prices at the peak, normal, and valley

periods respectively;

[114] 2) state set

[115] SDERi [SDERi I S'ER, I S ER (1

[116] In the formula, sDER,iis the xth state of the ith distributed resource, and4 is a number of states of the resource;

[117] 3) incentive coefficient data set

[118] CUL8= 5'~ U] (15) ct-= g, U [gin I gax]

[119] In the formula, 'oin and-ax are the minimal value and the maximum value of

the adjusted profits, wherein the minimum value is greater than 0, i.e., min >0, the

g.x< ' Cou(t)

[120] The DDQN algorithm mainly includes an evaluation network and a target network; during the off-line training, on one hand, action samples are recorded and stored in an experience pool D; on the other hand, by updating network parameters, the distributed resources can obtain optimal feedback behaviors by calculating the incentive coefficient; at the off-line phase, the parameters of the evaluation network are completely the same as those of the target network, and its algorithm is as follows:

[121] Algorithm 1 Training strategy for DERs 1 Initialized C, , C, , C,, store in set C, 2 For c=1 in C,, , do 3 Initialized , set the incentive range [ ' ,mS

4 Obtain the state set of DER SDER.

For s=1 in SDERI do 6 Set 8=8,a , While 5 < 5_ , do 7 Obtain the cost function of current state C 1 8 Estimate the best action a, and calculate the reward r, 9 Update the network according to r, Storethetrajectory r. =(s,,8,a,,s,) inD 11 =-+AS 12 End For

13 Reset the Initialized state s, = SERi 14 End for

[122] After the distributed resources run online, the two networks will run asynchronously, wherein the evaluation network runs online to constantly update the

network parameters and calculate an optimal behavior a,; the target network evaluates

value of the behavior, and the evaluation network copies the parameter value to the target network once in a while, and its algorithm is as follows:

[1231 Algorithm 2 Reasoning strategy for DERs'response 1 Received the stimulus message from VPP-CSC I 2 Obtain evaluation network Q, and target network Q2

3 Get current state St

4 Collect required parameters of network: electricity price Ct,,(t), state set SDEI?, , incentive coefficient 8(t), adjust cost function cad(se t For time step = is : tE do

6 Calculates action-value by Qt

7 Q +a(r+yQ2(s,',arg *(s,,at)<-Qi(st,a) maxaQi(s',ar))-Qi(st,at))

8 Update Qt network parameters by Q2 9 Q2 (st,at)<- Q 2 (st, at)+ a(r +yQi(st',argrmaxa Q2(st',a))- Q 2 (st, at))

Calculate ar=argmax AQ(st,at;O) considering user preference

11 End For 12 Get the best action sequence{ at }and power sequence { p, 13 Convert power sequence { p} to a message and response

[124] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in combination with the embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a proper manner.

[125] The preferred embodiments of the present disclosure disclosed above are only used to help illustrate the present disclosure. The preferred embodiment does not describe all the details, nor limits the present disclosure to the described specific embodiments only. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present disclosure, so that those skilled in the art can well understand and make use of the present disclosure. The present disclosure is only limited by the claims and their full scopes and equivalents.

Claims

WHAT IS CLAIMED IS:

1. A virtual power plant self-optimization load track control method, wherein comprising the following steps: Si: setting an evaluation index of self-optimization track control of a virtual power plant; S2: establishing an event-driven stimulation-feedback control mechanism; S3: realizing a DDQN-based training and decision-making method.

2. The virtual power plant self-optimization load track control method according to claim 1, wherein step Si is specially: in the load track control process, the virtual power plant coordinates distributed resources to increase or reduce load to achieve online matching of an expected target; since the virtual power plant provides external flexible services, it will obtain certain profits from external sources, such as power grids, power markets, and new energy operators, and meanwhile it needs to give a certain amount of compensation to the distributed resources that contribute internally; as for internal and external needs, defining following two indicators: Indicator 1: load track error (' AP t= ( t) t

,it={11,2,---, T}

Wherein: P,(t) and are actual load and target load of the virtual power

plant at time period of t, APL (t)is a time error of the time period; when AL()0, the

virtual power plant expects to reduce the load; otherwise, if AL(t)<0, the virtual power plant expects to increase the load; T is a number of time periods of the research cycle; Indicator 2: subsidy cost

F(A)= A,(t)AP

[8(t)=A, (t) / APL(t) (2)

wherein: A(t) is a subsidy amount released by the virtual power plant at the time period, with a unit of (RMB) Yuan; the final subsidy uses a linear allocation manner, i.e., the profits obtained by any unit load change is an incentive coefficient 85; for calculation of the indicators, using a logical electricity meters in the system assumption for aggregate statistics and settlement; constraints that need to be considered at the virtual power plant level including upper and lower constraints of the track ability, a climbing rate, and the network constraints in the distribution network environment.

3. The virtual power plant self-optimization load track control method according to claim 1, wherein in step S2, in the load track control scenario, the events generated by the virtual power plant mainly include the following two: 1) track events, TE; 2) coordinated events, CE

4. The virtual power plant self-optimization load track control method according claim 1, wherein the establishing an event-driven stimulation-feedback control mechanism mainly includes the steps of: Step 1: a load bias triggers the TE; defining that a bias between the load at some period and the target load satisfy formula (3), trigger the TE; otherwise, remove the TE;

FAP (t)d>9 Pgt) P(gt)t)>Qh | AP (t) > c, P,, (t! Q, (h

In the formula,d is proportion coefficient,and isanerrorconstant. Step 2: a virtual power plant cloud service center, i.e., VPP-CSC, releases an initial stimulus signal; in order to guide the distributed resources to adjust to the established direction, after the TE is triggered, the VPP-CSC will release a four-tuple stimulus I, (t) signal '" , e.g.:

I,,(t=[IS,, (t), t" , 1 4

In the formula: S, is task code; according to assumption, in the initial state, the VPP-CSC is unable to accurately know a number and state of the online distributed resources and actions taken possibly;

thus, an initial incentive value n (t) is obtained based on the historical load data;

{{J ,,(j)},P ,(t),AP inaf,., t)

llj= - -(5)

in the formula: 11 is a historical load of the virtual power plant in the past jth period; d is a referenced number of historical data; Step 3: the distributed resource feeds back adjustment plans according to stimulus signals, makes decision independently after receiving the stimulus signals, and feeds back the adjustment plans within the task period, i.e., load increasing (reducing) timing sequence:

PDER,i(Sc)= 1 '''>DER,i n (6),i

in the formula, PER,i(t) is the load increasing (reducing) amount of the ith distributed resource at the tthperiod; Step 4: the VPP-CSC determines whether to trigger the CE; before an expiration timestamp, counting a cumulative response amount fed back by the distributed resources, and triggering the CE if the cumulative amount satisfies formula (7): N (t)

P7,(t)= PDER W

P,(t)- AP( d>Qgt) gt(th

Pj(t) -AP (t) >c E~t> Cth (7)

in the formula, N(t) is a number of online distributed resources at period t,

is a cumulative response time of the distributed resource feedback at the period; Step 5: iterating until the CE is removed: ,5(t)=o(t)+oVD-(N(t)) (8)

in the formula, (t) is an incentive coefficient generated at the hth iteration at

period t, c is a predefined gradient step; Dh-1 (N(t))is a deviation distance after h

times of iteration when the terminal number is N), until (t) removes the CE, the distributed resource will add the coordination result into the task list again, and the CE removing condition is formulas (9)-(11): o., : of (t): oma g (9)

PI,(t) = AP - dtRpp ()inIt P,,, (t) =PA(t)=g -oc, (t)

P17 (t =AP,(t + 9, t)P1, (t)=g(g..,t) |P7 (t)= AP(t)+ d(

in the formula: g( 5,t) is a total response amount of the distributed resource

feedback when the incentive coefficient is 5 at period t, '5min and 'max are the theoretical minimum and maximum incentive coefficients to remove the error; Step 6: removing the TE; the distributed resource is self-regulated according to the task list, and the VPP-CSC determines whether the system satisfies the condition for removing the TE according to the check result, and performs a final calculation according to the verification result afterwards, the condition for removing the TE is:

FJp'(t)- '1(0t(t) P 7 (t) eg(t) m

intheformula, "(t is an actual track load of the integral virtual power plant after the distributed resource self-regulation is finished.

5. The virtual power plant self-optimization load track control method according to claim 1, wherein the step S3 is specially: using the DDQN algorithm to adaptively optimize the decision-making process of the distributed resources, the training method being: for the feedback training of the distributed resources, since the load track process of the virtual power plant has nothing to do with the online quantity, status and characteristics of the distributed resources, for any load increasing or decreasing amount APL(t) , the virtual power plant will adjust the incentive coefficient 6(t) to realize

accurate track of targets and optimization of control costs, and according to the principle of linear profit distribution, the distributed resources are essentially based on the income of unit load adjustments to make decisions; thus, the purpose of training is to build a network, establishing a standard mapping connection between stimulus-state-behavior based on the action value function; the data set of the training process includes: 1) time-of-use electricity price data set

C= [C,,,C,,,C"',,] (13) in the formula, v, fand p denote peak, normal, and valley periods respectively,

C,, Ci , and C"o are time-of-use electricity prices at the peak, normal, and valley periods respectively; 2) state set

SDERi=[sDERIisDER,, DERi

in the formula, sER,i is the xth state of the ith distributed resource, and is a number of states of the resource; 3) incentive coefficient data set

C ,=8'~ U[8 , ] (15)

In the formula, 5oin andn--ax are the minimal value and the maximum value of the

adjusted profits, wherein the minimum value is greater than 0, i.e., (min >0, the maximum value is smaller than ",times of the current time-of-use electricity price, i.e., gma < Ae C,-(t).

wherein the DDQN algorithm mainly includes an evaluation network and a target network; during the off-line training, on one hand, action samples are recorded and stored in an experience pool D; on the other hand, by updating network parameters, the distributed resources can obtain optimal feedback behaviors by calculating the incentive coefficient; at the off-line phase, the parameters of the evaluation network are completely the same as those of the target network; after the distributed resources run online, the two networks will run asynchronously, wherein the evaluation network runs online to constantly update the network parameters and calculate an optimal behavior

at; the target network evaluates Q value of the behavior, and the evaluation network

copies the parameter value to the target network once in a while.

－1/3－

FIG. 1

－2/3－

FIG. 2

－3/3－

FIG. 3