CN115986748A - Data-driven microgrid voltage control method - Google Patents
Data-driven microgrid voltage control method Download PDFInfo
- Publication number
- CN115986748A CN115986748A CN202211708121.2A CN202211708121A CN115986748A CN 115986748 A CN115986748 A CN 115986748A CN 202211708121 A CN202211708121 A CN 202211708121A CN 115986748 A CN115986748 A CN 115986748A
- Authority
- CN
- China
- Prior art keywords
- network
- voltage
- model
- power
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004146 energy storage Methods 0.000 claims abstract description 36
- 230000004888 barrier function Effects 0.000 claims abstract description 26
- 230000009471 action Effects 0.000 claims abstract description 22
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims description 24
- 239000003795 chemical substances by application Substances 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 13
- 238000012614 Monte-Carlo sampling Methods 0.000 claims description 8
- 230000008901 benefit Effects 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000002940 Newton-Raphson method Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- UFHFLCQGNIYNRP-UHFFFAOYSA-N Hydrogen Chemical compound [H][H] UFHFLCQGNIYNRP-UHFFFAOYSA-N 0.000 claims 1
- 229910052739 hydrogen Inorganic materials 0.000 claims 1
- 239000001257 hydrogen Substances 0.000 claims 1
- 230000006978 adaptation Effects 0.000 abstract 2
- 239000000243 solution Substances 0.000 description 15
- 230000001276 controlling effect Effects 0.000 description 5
- 230000000875 corresponding effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000010248 power generation Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 239000002918 waste heat Substances 0.000 description 1
Images
Landscapes
- Supply And Distribution Of Alternating Current (AREA)
Abstract
The invention discloses a data-driven microgrid voltage control method, which relates to the technical field of microgrid application and comprises the following steps: dividing the microgrid into a plurality of sub-networks which are mutually connected and coupled through tide based on a distributed architecture, and constructing an agent corresponding to the internal control of the sub-networks aiming at each sub-network; constructing a combined control model with the reactive output of the photovoltaic inverter as a main part and the active output of the energy storage system as an auxiliary part, and constructing a barrel-shaped voltage barrier function model by combining a V-shaped voltage barrier function model and a U-shaped voltage barrier function model; taking voltage constraint based on the network total power loss and the voltage barrier function model as target control, balancing voltage deviation and network power loss by using a weighting sum algorithm, and finally establishing a micro-grid distributed VVC model; the micro-grid distributed VVC model is fitted into a partially observable Markov decision POMG model, and a state action equation is modified from adaptation of discrete action to adaptation of continuous action so as to adapt to the real-time control requirement.
Description
Technical Field
The invention relates to the technical field of micro-grid application, in particular to a data-driven micro-grid voltage control method.
Background
A micro-grid (micro-grid) refers to a small-sized power generation and distribution system formed by collecting a distributed power supply, an energy storage device, an energy conversion device, and related load, monitoring and protection devices, and is an autonomous system capable of realizing self-control, protection and management, and can be operated in a grid-connected manner with an external power grid or in an isolated manner. Is an important component of the smart grid. The microgrid has a dual role: for a power grid, a micro-grid is used as an intelligent load with changeable size, a schedulable load is provided for a local power system, a response can be made within a few seconds to meet the system requirement, and powerful support is provided for the large power grid in due time; the system can be maintained without influencing the load of a client; the method can reduce (prolong) updating of the power distribution network, adopts the IEEE1547.4 standard to guide the operation of the distributed power supply island, and can eliminate technical obstacles generated by certain special operation requirements. For the user, the microgrid acts as a customizable power supply, which can meet the diverse needs of the user, for example, enhancing local power supply reliability, reducing feed losses, supporting local voltage, improving efficiency by utilizing waste heat, providing correction of voltage sag, or serving as an uninterruptible power supply, etc. The micro-grid not only solves the problem of large-scale access of the distributed power supply, gives full play to various advantages of the distributed power supply, but also brings other multi-aspect benefits to users. The micro-grid fundamentally changes the traditional mode of coping with load increase, and has great potential in the aspects of reducing energy consumption, improving reliability and flexibility of a power system and the like.
For micro-grids, some voltage/reactive power control (VVC) methods have been developed to solve the voltage regulation problem and reduce the power consumption of the grid and other power management problems. Among other things, inverters and energy storage devices have been extensively studied and used to participate in microgrid regulation control due to their increasing availability, flexibility and rapid response speed of advanced power electronics technologies. The control framework for the microgrid mainly comprises 3 control frameworks of centralized control, local control and distributed control at present. Distributed methods have been increasingly studied because they do not require complex communication networks and can take into account the privacy, complexity, scalability, and globality issues that exist.
A common research idea for distributed control is to transform a non-convex problem into a convex optimization problem through reasonable approximation and simplification, and then solve the problem using a distributed algorithm. To cope with the uncertainty of distributed new energy and load demand, model-based distributed control methods typically require a predetermined optimal solution. However, when new situations arise, conventional model control methods must solve optimization problems, which require a large amount of computation. Furthermore, model-based control methods typically require accurate and comprehensive grid parameters, which are often difficult to achieve.
Disclosure of Invention
In order to solve the above-mentioned drawbacks of the prior art, the present invention provides a data-driven microgrid voltage control method.
The purpose of the invention can be realized by the following technical scheme: a data-driven microgrid voltage control method comprises the following steps:
dividing a micro-grid into a plurality of sub-networks which are mutually connected and coupled through power flow on the basis of a distributed architecture, and constructing an intelligent agent corresponding to the internal control of the sub-networks for each sub-network;
constructing a combined control model with the reactive output of the photovoltaic inverters as a main part and the active output of the energy storage system as an auxiliary part, controlling all the photovoltaic inverters and energy storage equipment in the sub-networks by a single intelligent body corresponding to each sub-network, and realizing effective control on the voltage of the micro-grid by controlling the reactive output of the photovoltaic inverters and the active output of the energy storage system;
combining the V-shaped voltage barrier function model and the U-shaped voltage barrier function model to construct a barrel-shaped voltage barrier function model;
taking voltage constraint based on a network total power loss and voltage barrier function model as target control, balancing voltage deviation and network power loss by using a weighting sum algorithm, establishing a space-time uncertainty model based on photovoltaic and load prediction intervals, establishing a network power flow constraint model based on Newton-Raphson method solution, integrating the space-time uncertainty model based on the photovoltaic and load prediction intervals, a combined control model taking the reactive power output of a photovoltaic inverter as a main part and taking the active power output of an energy storage system as an auxiliary part, balancing a control target after voltage deviation and network power loss and the network power flow constraint model, and establishing a micro-grid distributed VVC model;
fitting a micro-grid distributed VVC model into a partially observable Markov decision POMG model, and modifying a state action equation from adaptive discrete action to adaptive continuous action to adapt to real-time control requirements;
converting the constructed space-time uncertainty model based on the photovoltaic and load prediction interval into a random planning model, and adding a network solving failure punishment in the return reward of each training set to improve the network solving success rate;
a multi-agent depth certainty strategy gradient MADDPG algorithm is used, an algorithm process is designed, and the algorithm process is applied to a micro-grid.
Preferably, the distributed architecture does not require a central coordinator to collect all the information generated in the network, requiring global coordination issues to be considered; in distributed optimization, the sub-network represented by each constructed agent exchanges only limited boundary physical information and global reward return with its neighbors, collectively seeking a global optimal solution, during each operation the trained controller implements locally measured VVC within the respective sub-network.
Preferably, the joint control model with the photovoltaic inverter reactive output as the main and the energy storage system active output as the auxiliary is as follows:
The photovoltaic inverters preferentially provide reactive power when needed, when the reactive power compensation capability is insufficient, the energy storage system can act, the reactive power output by each photovoltaic inverter is limited in a preset proportion of the apparent power capacity, a positive value indicates that the reactive power is injected into a power grid, and a negative value indicates that the reactive power of the power grid is absorbed; the energy storage system is arranged similarly to the inverter, positive values indicate that active power is injected into the power grid, negative values indicate that the active power of the power grid is absorbed, and the residual electric quantity of the energy storage system is always positive.
Preferably, the barrel-type voltage barrier function model is as follows:
in the formula, v a Real-time voltage magnitude for the node; v. of ref Taking 1.00p.u. as a network voltage reference value; l v (v a ) Real-time reward for the node voltage;
and the voltage barrier function model combines the advantages of V-shaped and U-shaped: on one hand, it has a slow gradient in a safe range, and better voltage conditions are obtained; on the other hand, a larger gradient outside the safe range ensures faster policy guidance.
Preferably, the control target after balancing the voltage deviation and the network power loss is as follows:
in the formula, lv (vi, t) is a real-time voltage barrier function value of an i node at the t moment; n is the number of network nodes; n is a radical of m Is a network node set;
is a network branch set; r is ij And x ij Respectively representing the resistance and reactance of a branch between the i node and the j node; v. of i,t And v j,t Respectively representing the voltage amplitudes of an i node and a j node at the time t;
then, a weighting sum algorithm is adopted to convert the multi-objective function into an equivalent single objective function with weighting factors, the targets are normalized by using Utox points and nadier points, and the weighting sum of the normalized targets obtained for any sub-network m is expressed as:
Preferably, the process of establishing the photovoltaic and load prediction interval-based spatio-temporal uncertainty model is as follows:
before each operation period, generating a spatial uncertainty scene in a given prediction interval randomly through Monte Carlo sampling, generating a time uncertainty scene through Monte Carlo sampling in each operation period, considering time lag through the time uncertainty interval, calculating voltage deviation and network loss under each time uncertainty scene, and correcting the two targets by using scene occurrence probability, wherein the corrected normalized target is represented as:
Preferably, the process of modifying the state action equation from adapting to discrete action to adapting to continuous action is as follows:
the state-action function is used to indicate the current state or the integrated return of a state-action pair as follows:
in the formula, τ i Represents a history of agent i; a is -i ＝× j≠i a j To accommodate continuous motion, the state-motion function is modified to the form:
in the formula (II), based on a' i Is distributed by pi i (a′ i ∣τ i ) It is shown that,
obtained by monte carlo sampling, as:
preferably, the process of adding a network solution failure penalty to increase the network solution success rate is as follows:
adding a network solution failure penalty function F:
F＝-f,t f ＜T max
in the formula, f is a penalty occurrence constant and is a large positive number; t is t f The duration of successful network solving in the training set with interruption due to failure of network solving; t is max Maximum time step for each training set;
the reward for each training set interrupted by the occurrence of network solution failure is adjusted to R mf ：
R mf ＝R m +F
In the formula, R m The normal prize value obtained before the training set break.
Preferably, an apparatus comprises:
one or more processors;
a memory for storing one or more programs;
when executed by one or more of the processors, cause the one or more processors to implement a data-driven microgrid voltage control method as described above.
Preferably, a storage medium contains computer executable instructions for performing a data driven microgrid voltage control method as described above when executed by a computer processor.
The invention has the beneficial effects that:
based on a data driving method and a distributed architecture, the invention adopts a mode of combining centralized training and decentralized execution, and obtains an effective operation control strategy by performing offline training on historical data, thereby being capable of making a real-time joint output decision on line according to the real-time operation condition of the network, and ensuring the reliability, economy and safety in the network operation process.
Drawings
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present invention, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without creative efforts;
FIG. 1 is a schematic diagram of the overall framework of the network of the present invention;
FIG. 2 is a schematic diagram of an algorithm flow framework of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, a method for controlling voltage of a data-driven microgrid comprises the following steps:
step (1): the microgrid is divided into a plurality of sub-networks which are mutually connected and coupled through tide based on a distributed architecture, and an agent controlled inside the corresponding sub-network is constructed for each sub-network.
Considering that the traditional centralized VVC framework has the problems of communication and computation burden, privacy, and the like, and the problem that the traditional local control method cannot perform global coordination, a regional coordination microgrid VVC framework is constructed, as shown in fig. 1. The micro-grid is divided into a plurality of sub-networks which are mutually connected and coupled through power flow on the basis of a distributed architecture, and an agent corresponding to internal control of each sub-network is constructed. Each constructed agent's sub-network only exchanges limited boundary physical information and global reward return with its neighboring sub-networks, collectively seeking a global optimal solution. During each operation, the trained controller implements locally measured VVCs within its respective sub-network.
Step (2): and constructing a combined control model with the reactive output of the photovoltaic inverters as a main part and the active output of the energy storage system as an auxiliary part, controlling all the photovoltaic inverters and energy storage equipment in the sub-networks by a single intelligent body corresponding to each sub-network, and realizing effective control on the voltage of the micro-grid by controlling the reactive output of the photovoltaic inverters and the active output of the energy storage system.
The inverter control model is shown in (1) to (3). The reactive power output by each inverter is also limited within a preset proportion of the apparent power capacity of the inverter, positive values indicate that the reactive power is injected into the power grid, and negative values indicate that the reactive power of the power grid is absorbed.
Wherein,
represents the real-time active power of the photovoltaic output->
Represents the real-time reactive power output by the photovoltaic inverter and is greater or less>
Representing photovoltaicsThe complex power of the inverter is->
And a photovoltaic active power output boundary at t, and a delta photovoltaic inverter reactive power capacity factor are represented.
The photovoltaic inverter droop control scheduling model is shown as (4) and (5). The energy storage system is arranged similarly to the inverter, positive values indicate that active power is injected into the power grid, negative values indicate that the active power of the power grid is absorbed, and the residual electric quantity of the energy storage system is always positive.
Wherein,
and &>
Represents the minimum and maximum active power that the energy storage system can emit and absorb->
Represents the active power output value of the stored energy at t->
Represents the real-time capacity of the stored energy at t->
Representing the maximum capacity of stored energy.
The photovoltaic inverters preferentially provide reactive power when needed, when the reactive power compensation capability is insufficient, the energy storage system can act, the reactive power output by each photovoltaic inverter is limited in a preset proportion of the apparent power capacity, a positive value indicates that the reactive power is injected into a power grid, and a negative value indicates that the reactive power of the power grid is absorbed; the energy storage system is arranged similarly to the inverter, positive values indicate that active power is injected into the power grid, negative values indicate that the active power of the power grid is absorbed, and the residual electric quantity of the energy storage system is always positive.
And (3): combining the V-shaped voltage barrier function model and the U-shaped voltage barrier function model to construct a barrel-shaped voltage barrier function model;
in the formula (6), the V-shaped voltage barrier function has a larger gradient in the range of 0.95p.u. -1.05p.u., so that better voltage conditions can be realized, but fine adjustment cannot be realized in the range. The U-shaped voltage barrier function in the formula (7) has a small gradient outside the range of 0.95p.u. -1.05p.u, and cannot enter a safety range quickly.
Combining the advantages of the two types of the voltage barrier function model:
in the formula (8), v a The real-time voltage magnitude of the node; v. of ref Generally, 1.00p.u. is taken as a network voltage reference value; l v (v a ) Real-time awards for the node voltage.
The model combines the advantages of V-shaped and U-shaped models: on one hand, the gradient of the voltage gradient is slow in a safe range, so that better voltage conditions can be obtained; on the other hand, a larger gradient outside the safe range may ensure faster policy guidance.
And (4): the method comprises the steps of taking voltage constraint based on a network total power loss and voltage barrier function model as target control, balancing voltage deviation and network power loss by using a weighting sum algorithm, establishing a space-time uncertainty model based on photovoltaic and load prediction intervals, establishing a network power flow constraint model based on Newton-Raphson method solution, integrating the space-time uncertainty model based on the photovoltaic and load prediction intervals, taking the reactive power output of a photovoltaic inverter as a main combined control model and taking the active power output of an energy storage system as an auxiliary combined control model, balancing a control target after voltage deviation and network power loss and the network power flow constraint model, and establishing a micro-grid distributed VVC model;
two VVC targets, i.e., voltage constraint and network active power minimization, are set per subnet as shown in (9). Where (10) is e.g. the voltage deviation and (11) the total power loss at time t of the network.
In the formula (10), l v (v i,t ) The value of the real-time voltage barrier function of the i node at the time t; n is the number of network nodes; n is a radical of m Is a network node set; in formula (11)
For a set of network legs, r ij And x ij Representing the resistance and reactance of the branch between the i and j nodes, respectively. v. of i,t And v j,t Respectively, the voltage amplitudes of the i node and the j node at time t.
And converting the multi-objective function into an equivalent single objective function with a weighting factor by adopting a classical weighted sum algorithm. Here we normalize these targets using the utopia point and nadir point, and for subnetwork m we get a weighted sum of the normalized targets expressed as:
in the formula,
represents the voltage deviation at the moment t of the m-network after normalization, and>
and the network active loss at the t moment of the m network after normalization is represented, and alpha and beta represent normalized coefficients.
And establishing a space-time uncertainty model based on photovoltaic and load prediction intervals.
A spatial uncertainty model is first established based on the prediction interval of photovoltaic power generation and load, and then optimized at each decision time step. The time interval in the optimized model varies with time and is correlated with the real-time measurement at t.
Spatial and temporal uncertainties need to be addressed to ensure operational constraints, taking into account the location of renewable energy generation and load changes, as well as short term intermittency and fluctuations.
Firstly, a space uncertainty model is established based on the predicted photovoltaic power generation time interval and load, and the uncertainty which can be realized at the position t0 is limited in a prediction interval:
in the formula,
and representing the upper and lower limits of active and reactive power requirements of the maximum power point/bus i of the space uncertainty inverter.
In the above spatial uncertainty model, communication delay may cause response delay of the inverter, and in order to eliminate the influence of these delays, a time uncertainty model is established, and each decision time step is optimized and expressed as:
in the formula,
and (3) showing the time uncertainty upper and lower limits of the maximum power point/bus active power and reactive power requirements of the inverter. The spacing of this time uncertainty model varies with time and is related to the real-time measurement at t.
The intervals of the time uncertainty model vary with time and are related to real-time measurements at t time.
And establishing a network power flow constraint model based on Newton Raphson method solution as shown in a formula (18).
In the formula p i,t And q is i,t Respectively representing active power and reactive power injected by the i node at the moment t, v i,t Representing the magnitude of the voltage at time t of the i-node, g ij And b ij Representing the conductance and susceptance, θ, of the busbar between the i-node and the j-node ij,t And N is an index of all nodes of the microgrid, and represents the voltage phase angle difference between the i node and the j node at the time t.
And (3) integrating a space-time uncertainty model based on photovoltaic and load prediction intervals, a combined control model with photovoltaic inverter reactive power output as a main part and energy storage system active power output as an auxiliary part, a balanced voltage deviation, a network power loss control target and a network power flow constraint model, and establishing a distributed VVC model of the microgrid as shown in a formula (19).
In the formula (19), M is a network partition index; t is the total duration; v. of i,t Is a real-time voltage.
And (5): fitting a micro-grid distributed VVC model into a partially observable Markov decision POMG model, and modifying a state action equation from adaptive discrete action to adaptive continuous action to adapt to real-time control requirements;
firstly, setting an observation space containing v i,t 、
And &>
Respectively representing node real-time voltage, active power consumption of a node connection load, reactive power consumption of the node connection load, active power of a real-time photovoltaic injection power grid and real-time energy storage residual energy; the successive action spaces comprising->
And &>
And respectively representing the reactive output power and the energy storage active output power of the inverter.
The goal of determining each actor is to maximize its expected revenue over a time horizon T, i.e.
The state-motion equations are modified from adapting discrete motion to adapting continuous motion to accommodate real-time control requirements.
The state-action function is used to indicate the current state or the integrated return of a state-action pair:
in the formula, τ i Represents a history of agent i; a is -i ＝× j≠i a j . To accommodate the continuous motion, we modify it to the following form:
in the formula (II), based on a' i Is distributed by pi i (a′ i ∣τ i ) And (4) showing. In the practical application of the method, the material is,
obtained by approximation of a monte carlo sample, which can be rewritten as:
and (6): converting the constructed space-time uncertainty model based on the photovoltaic and load prediction interval into a random planning model, and adding a network solving failure punishment in the return reward of each training set to improve the network solving success rate;
a spatial uncertainty scene is generated by Monte Carlo sampling according to the prediction intervals of equations (13) and (14), a time uncertainty scene is generated by Monte Carlo sampling according to equations (15) and (16) in consideration of time lag, and the voltage deviation and the network loss are corrected by using the scene occurrence probability. Xi shape u The occurrence probability of the situation U belongs to U, and the situation is normalized:
in equation (24), U considers each decisionThere are U time uncertainty scenes xi on the time step u Is the probability of occurrence of the scenario U e U.
The sum of the values (average voltage deviation/grid loss at time instant m network t) is normalized for the case in stochastic programming.
Is the sum of the normalized values in the case of u.
Finally, a strategy gradient function is given:
in order to reduce the problem of network solution failure in the training process caused by the problem of the bearing capacity of the power network, a network solution failure penalty function F is added:
F＝-f,t f ＜T max (25)
in the formula, f is a penalty occurrence constant and is a larger positive number; t is t f The duration of successful network solving in the training set with interruption due to failure of network solving; t is max The maximum time step for each training set.
Then the reward for each training set interrupted by the occurrence of network solution failure at this time is adjusted to:
R mf ＝R m +F(26)
in the formula (26), R m The normal prize value obtained before the training set interruption.
And (7): a multi-agent depth certainty strategy gradient MADDPG algorithm is used, a reasonable algorithm flow is designed, and the method is applied to a micro-grid.
The specific algorithm flow is shown in fig. 2. During initialization, the capacity of the energy storage system is set to be 50% of the total capacity, and if the energy storage action is carried out, the action value same as that of the photovoltaic is adopted. For each training set, the buffer stores 480 time steps (i.e., 1 day) of PV and load data. Further, the bonus is obtained based on the calculation result of the buffer data according to the PandaPower package. The received state assigns a zone according to each agent to an observation before providing feedback to the agent. Each agent receives only local observations and global rewards before making the next decision. The above process will be repeated until the training single set is finished.
After each set of actions is completed, a reward is obtained. A training comprises a plurality of training sets, and the number of sets required by training is manually set according to needs. Behavior change and policy model preservation occurs once every 40 training sets and the targets are updated once every 120 sets. Before each policy model update, 10 sets of tests will be performed. The test data is based on the total data sample and the average of the test will be used to evaluate the effectiveness of the strategy.
Based on the same inventive concept, the present invention also provides a computer apparatus, comprising: one or more processors, and memory for storing one or more computer programs; the program includes program instructions and the processor is configured to execute the program instructions stored by the memory. The Processor may be a Central Processing Unit (CPU), or may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc., which is a computing core and a control core of the terminal and is used for implementing one or more instructions, and in particular for loading and executing one or more instructions in a computer storage medium, so as to implement the method described above.
It should be further noted that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the above method. The storage medium may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
In the description herein, references to the description of "one embodiment," "an example," "a specific example," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the disclosure. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing illustrates and describes the general principles, principal features, and advantages of the present disclosure. It will be understood by those skilled in the art that the present disclosure is not limited to the embodiments described above, which are presented solely for illustrating the principles of the disclosure, but that various changes and modifications may be made without departing from the spirit and scope of the disclosure, which fall within the scope of the claimed disclosure.
Claims (10)
1. A data-driven microgrid voltage control method is characterized by comprising the following steps:
dividing the microgrid into a plurality of sub-networks which are mutually connected and coupled through tide based on a distributed architecture, and constructing an agent corresponding to the internal control of the sub-networks aiming at each sub-network;
constructing a combined control model with the reactive output of the photovoltaic inverters as a main part and the active output of the energy storage system as an auxiliary part, controlling all the photovoltaic inverters and energy storage equipment in the sub-networks by a single intelligent body corresponding to each sub-network, and realizing effective control on the voltage of the micro-grid by controlling the reactive output of the photovoltaic inverters and the active output of the energy storage system;
combining the V-shaped voltage barrier function model and the U-shaped voltage barrier function model to construct a barrel-shaped voltage barrier function model;
taking voltage constraint based on a network total power loss and voltage barrier function model as target control, balancing voltage deviation and network power loss by using a weighting sum algorithm, establishing a space-time uncertainty model based on photovoltaic and load prediction intervals, establishing a network power flow constraint model based on Newton-Raphson method solution, integrating the space-time uncertainty model based on the photovoltaic and load prediction intervals, a combined control model taking the reactive power output of a photovoltaic inverter as a main part and taking the active power output of an energy storage system as an auxiliary part, balancing a control target after voltage deviation and network power loss and the network power flow constraint model, and establishing a micro-grid distributed VVC model;
fitting a micro-grid distributed VVC model into a partially observable Markov decision POMG model, and adapting to real-time control requirements by improving a state action equation from adapting to discrete actions to adapting to continuous actions;
converting the well-constructed space-time uncertainty model based on the photovoltaic and load prediction interval into a stochastic programming model, and adding a network solution failure punishment in the return reward of each training set to improve the success rate of network solution;
a multi-agent depth certainty strategy gradient MADDPG algorithm is used, an algorithm process is designed, and the algorithm process is applied to a micro-grid.
2. The voltage control method of the data-driven microgrid according to claim 1, characterized in that the distributed architecture does not require a central coordinator to collect all the information generated in the network, and needs to consider global coordination problems; in distributed optimization, the sub-network represented by each constructed agent exchanges only limited boundary physical information and global reward return with its neighbors, collectively seeking a global optimal solution, during each operation the trained controller implements locally measured VVC within the respective sub-network.
3. The method for controlling the voltage of the data-driven microgrid according to claim 1, wherein the combined control model with the photovoltaic inverter having the reactive output as the main and the energy storage system having the active output as the auxiliary is as follows:
the photovoltaic inverters preferentially provide reactive power when needed, when the reactive power compensation capability is insufficient, the energy storage system can act, the reactive power output by each photovoltaic inverter is limited in a preset proportion of the apparent power capacity, a positive value indicates that the reactive power is injected into a power grid, and a negative value indicates that the reactive power of the power grid is absorbed; the energy storage system is arranged similarly to the inverter, positive values indicate that active power is injected into the power grid, negative values indicate that the active power of the power grid is absorbed, and the residual electric quantity of the energy storage system is always positive.
4. The method of claim 1, wherein the bucket type voltage barrier function model is as follows:
in the formula, v a The real-time voltage magnitude of the node; v. of ref Taking 1.00p.u. as a network voltage reference value; l v (v a ) Real-time reward for node voltage;
and the voltage barrier function model combines the advantages of the V type and the U type: on one hand, it has a slow gradient in a safe range, and better voltage conditions are obtained; on the other hand, a larger gradient outside the safe range ensures faster policy guidance.
5. The method of claim 1, wherein the trade-off between voltage deviation and network power consumption is the control objective:
in the formula I v (v i,t ) The value of the real-time voltage barrier function of the i node at the time t; n is the number of network nodes; n is a radical of hydrogen m Is a network node set;
is a network branch set; r is ij And x ij Respectively representing the resistance and reactance of a branch between the i node and the j node; v. of i,t And v j,t Respectively represent t timesVoltage amplitudes of an i node and a j node are carved;
then, a weighting sum algorithm is adopted to convert the multi-objective function into an equivalent single objective function with weighting factors, the targets are normalized by using Utox points and nadier points, and the weighting sum of the normalized targets obtained for any sub-network m is expressed as:
6. The data-driven microgrid voltage control method of claim 1, characterized in that the process of establishing a photovoltaic and load prediction interval-based spatio-temporal uncertainty model is as follows:
before each operation period, generating a spatial uncertainty scene in a given prediction interval randomly through Monte Carlo sampling, generating a time uncertainty scene through Monte Carlo sampling in each operation period, considering time lag through the time uncertainty interval, calculating voltage deviation and network loss under each time uncertainty scene, and correcting the two targets by using scene occurrence probability, wherein the corrected normalized target is represented as:
7. The method of claim 1, wherein the process of modifying the equation of state action from adaptive discrete action to adaptive continuous action is as follows:
the state-action function is used to indicate the current state or the integrated return of a state-action pair as follows:
in the formula, τ i Represents a history of agent i; a is -i ＝× j≠i a j To accommodate continuous motion, the state-motion function is modified to the following form:
in the formula (II), based on a' i Is distributed by pi i (a′ i ∣τ i ) It is shown that,
obtained by monte carlo sampling, as:
8. the data-driven microgrid voltage control method according to claim 1, wherein the process of adding a network solution failure penalty to increase a network solution success rate is as follows:
adding a network solution failure penalty function F:
F＝-f,t f ＜T max
in the formula, f is a penalty occurrence constant and is a large positive number; t is t f The duration of successful network solving in the training set with interruption due to failure of network solving; t is a unit of max Maximum time step for each training set;
the reward for each training set interrupted by the occurrence of network solution failure is adjusted to R mf ：
R mf ＝R m +F
In the formula, R m The normal prize value obtained before the training set break.
9. An apparatus, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by one or more of the processors, cause the one or more processors to implement a data-driven microgrid voltage control method of any of claims 1-8.
10. A storage medium containing computer executable instructions for performing a data driven microgrid voltage control method according to any one of claims 1 to 8 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211708121.2A CN115986748A (en) | 2022-12-29 | 2022-12-29 | Data-driven microgrid voltage control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211708121.2A CN115986748A (en) | 2022-12-29 | 2022-12-29 | Data-driven microgrid voltage control method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115986748A true CN115986748A (en) | 2023-04-18 |
Family
ID=85973701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211708121.2A Pending CN115986748A (en) | 2022-12-29 | 2022-12-29 | Data-driven microgrid voltage control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115986748A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116581766A (en) * | 2023-07-11 | 2023-08-11 | 南京理工大学 | Virtual power plant strengthening online voltage control method considering sagging characteristic |
-
2022
- 2022-12-29 CN CN202211708121.2A patent/CN115986748A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116581766A (en) * | 2023-07-11 | 2023-08-11 | 南京理工大学 | Virtual power plant strengthening online voltage control method considering sagging characteristic |
| CN116581766B (en) * | 2023-07-11 | 2023-11-28 | 南京理工大学 | Virtual power plant strengthening online voltage control method considering sagging characteristic |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Conte et al. | A stochastic optimization method for planning and real-time control of integrated pv-storage systems: Design and experimental validation | |
| Bahramipanah et al. | Decentralized voltage control of clustered active distribution network by means of energy storage systems | |
| Venkatesan et al. | Optimal power flow control of hybrid renewable energy system with energy storage: A WOANN strategy | |
| CN114725936A (en) | Power distribution network optimization method based on multi-agent deep reinforcement learning | |
| CN105826944B (en) | A kind of power forecasting method and system of micro-capacitance sensor group | |
| Nguyen et al. | Three-stage inverter-based peak shaving and Volt-VAR control in active distribution networks using online safe deep reinforcement learning | |
| CN112651112B (en) | Collaborative decision-making method, system and equipment for electric energy transaction and system operation of internet micro-grid | |
| Xiang et al. | Deep reinforcement learning based topology-aware voltage regulation of distribution networks with distributed energy storage | |
| CN112952831B (en) | Daily optimization operation strategy for providing stacking service by load side energy storage | |
| Ameur et al. | Intelligent optimization and management system for renewable energy systems using multi-agent | |
| CN117833285A (en) | Micro-grid energy storage optimization scheduling method based on deep reinforcement learning | |
| CN115986748A (en) | Data-driven microgrid voltage control method | |
| Huang et al. | Distributed real-time economic dispatch for islanded microgrids with dynamic power demand | |
| Hossain et al. | Deep reinforcement learning-based volt-var optimization in distribution grids with inverter-based resources | |
| Ye et al. | A hybrid charging management strategy for solving the under-voltage problem caused by large-scale EV fast charging | |
| Taghizadeh et al. | Deep reinforcement learning-aided bidding strategies for transactive energy market | |
| Li et al. | Distributed control of energy-storage systems for voltage regulation in distribution network with high pv penetration | |
| Xiong et al. | Reactive power optimization in active distribution systems with soft open points based on deep reinforcement learning | |
| CN109672216A (en) | One kind being based on polyfactorial active distribution network hierarchical control method and system | |
| Pinthurat et al. | Unbalanced powers compensation by distributed batteries in microgrid: A data-driven approach | |
| Bonthu et al. | Energy cost optimization in microgrids using model predictive control and mixed integer linear programming | |
| CN117096962A (en) | Photovoltaic-considered power grid dynamic reactive power compensation optimization method and system | |
| Xu et al. | Distributed Reactive Power Optimization for Energy Internet via Multiagent Deep Reinforcement Learning With Graph Attention Networks | |
| Ebell et al. | Coordinated multi-agent reinforcement learning for swarm battery control | |
| Huang et al. | Multi-agent deep reinforcement learning with enhanced collaboration for distribution network voltage control |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |