CN116260143A - Distribution network switch automatic control method and system based on reinforcement learning theory - Google Patents

Abstract

The invention discloses a distribution network switch automatic control method and system based on reinforcement learning theory. The method includes: establishing a Distflow power flow optimization constraint model, determining a reinforcement learning algorithm for approximate dynamic programming; obtaining historical output data of distributed power generation units , and establish the output power fluctuation and conversion model of the distributed generation unit; determine the topology structure of the distribution network, obtain the active output data of the distributed generation unit, the controllable section switch of the distribution network, the status information of the tie switch and the calculation of the Distflow power flow optimization constraint model The result information, and the MDP model is established according to the output power fluctuation and conversion model of the distributed power generation unit, the distribution network topology and the obtained information; the reinforcement learning algorithm is used to solve the MDP model, and the optimal strategy for the automatic control of the distribution network switch is output in real time . The invention can solve the problems of fluctuations, failures and failures of the distributed power generation units in the power distribution network, improve the power supply reliability of the power distribution system, and increase the investment benefit of the power distribution system.

Description

Automatic control method and system of distribution network switches based on reinforcement learning theory

Technical Field

The present invention relates to the technical field of power grid dispatching, and in particular to a distribution network switch automatic control method and system based on reinforcement learning theory.

Background Art

The distribution network undertakes the arduous task of receiving and distributing electric energy in the power system. It is directly facing the electricity terminals and is closely connected with the daily production and life of the general public. The traditional distribution network has a "tree" structure. This "radiating" structure has many weaknesses: serious problems such as large fault damage, poor mutual supply capacity, and low degree of automation. Although the manufacturing cost is greatly reduced, its reliability is not high. In recent years, more and more distributed generation systems (DG) such as wind power generation and photovoltaic power generation have been connected to the distribution network. Such optimization strategies can meet the development requirements of the clean, environmentally friendly, low-cost, efficient and reliable power industry to a certain extent.

However, the access to these renewable energy sources has also had many adverse effects on the distribution network. The biggest problem is that new energy generation modes such as wind power generation and solar power generation will experience many random fluctuations due to environmental influences, and there is uncertainty and instability; secondly, due to natural disasters and man-made disasters, the nodes and branches of the distribution network are prone to failure. These adverse factors make the working state of the distribution system complex and changeable, directly affecting the safe operation of the distribution system.

Summary of the invention

The present invention aims to solve one of the technical problems in the related art to at least a certain extent. To this end, the first object of the present invention is to provide a distribution network switch automatic control method based on reinforcement learning theory, which can solve the working fluctuation, failure and fault problems of distributed power generation units in the distribution network, improve the power supply reliability of the distribution system, and improve the investment efficiency of the distribution system.

The second object of the present invention is to provide a distribution network switch automatic control system based on reinforcement learning theory.

To achieve the above object, the present invention is implemented through the following technical solutions:

A distribution network switch automatic control method based on reinforcement learning theory, comprising:

Step S1: Establish a Distflow power flow optimization constraint model and determine a reinforcement learning algorithm for approximate dynamic programming;

Step S2: Obtain historical output data of photovoltaic and wind power distributed generation units, and establish an output power fluctuation and conversion model of the distributed generation units according to the historical output data;

Step S3: determine the topological structure of the distribution network, obtain the active output data of the distributed generation units, the status information of the controllable sectional switches and the tie switches of the distribution network, and the calculation result information of the Distflow power flow optimization constraint model, and establish a distribution network Markov decision process MDP model according to the output power fluctuation and conversion model of the distributed generation units, the topological structure of the distribution network and the information obtained in step S3;

Step S4: using the reinforcement learning algorithm of approximate dynamic programming to solve the Markov decision process MDP model of the distribution network, and outputting the optimal strategy for automatic control of distribution network switches in real time.

Optionally, the step S1 includes:

Step S11: establishing the Distflow power flow optimization constraint model according to the distribution network topology constraints and the basic theory of distribution network power flow calculation;

Step S12: Determine the reinforcement learning algorithm of approximate dynamic programming according to the reinforcement learning theory and the Bellman optimal equation.

Optionally, in step S2, before establishing the output power fluctuation and conversion model of the distributed power generation unit, the method also includes: analyzing and quantifying the output power fluctuation and uncertainty conditions of each distributed power generation unit, so as to use the quantified value as input of the Markov decision process MDP model of the distribution network.

Optionally, the output power fluctuation and conversion model of the distributed power generation unit can simulate the volatility and uncertainty of the output power of each distributed power generation unit.

Optionally, step S3 includes:

Step S31: Modeling the output power fluctuation status quantization value of each distributed power generation unit and the corresponding distribution network topology structure as state parameters of the distribution network Markov decision process MDP model;

Step S32: Modeling the action states of switches connected to each branch of the power distribution network as action combination parameters of the Markov decision process MDP model of the power distribution network;

Step S33: Selecting the load shedding and line operation cost modeling in the Distflow power flow optimization constraint model considering distributed generation failure as the reward function reference index of the distribution network Markov decision process MDP model;

Step S34: defining the state transition probability of the Markov decision process MDP model of the power distribution network, so as to take into account the uncertainty caused by the probability of change of the power output level of each distributed generation unit.

Optionally, in step S4, before outputting the optimal strategy for automatic control of the distribution network switches in real time, the method further includes: performing offline iterative training on the Markov decision process MDP model of the distribution network.

Optionally, the step of performing offline iterative training on the Markov decision process MDP model of the distribution network includes: inputting active output data of distributed generation units, outputting automatic control strategies for distribution network switches, and feeding back evaluation indicators of real-time operation of the distribution network.

Optionally, in step S4, after outputting the optimal strategy for automatic control of the distribution network switches in real time, the method further comprises: displaying the decision result in text and the real-time distribution network topology structure on a human-computer interaction interface.

Optionally, the action state of the switch includes: the switch changes the switch state at the current moment and maintains the switch state at the current moment.

To achieve the above object, the second aspect of the present invention provides a distribution network switch automatic control system based on reinforcement learning theory, comprising:

Establish a module for establishing the Distflow power flow optimization constraint model;

A determination module, used to determine a reinforcement learning algorithm that approximates dynamic programming;

An acquisition module is used to acquire historical output data of photovoltaic and wind power distributed generation units, so that the establishment module can establish an output power fluctuation and conversion model of the distributed generation units according to the historical output data;

The establishment module is also used to establish a distribution network Markov decision process MDP model according to the distribution network topology determined by the determination module, the active output data of the distributed generation units obtained by the acquisition module, the controllable sectional switches of the distribution network, the status information of the tie switches and the calculation result information of the Distflow power flow optimization constraint model, and the output power fluctuation and conversion model of the distributed generation units;

The calculation module is used to solve the Markov decision process MDP model of the distribution network according to the reinforcement learning algorithm of approximate dynamic programming, and output the optimal strategy for automatic control of distribution network switches in real time.

The present invention has at least the following technical effects:

1. The present invention establishes a distributed power generation unit output power fluctuation and conversion model through historical output data of distributed power generation units such as photovoltaic and wind power, and obtains a Markov decision process MDP model through the distributed power generation unit output power fluctuation and conversion model. Since the distributed power generation unit output power fluctuation and conversion model fully considers the working efficiency volatility of new energy power generation modes such as wind power generation and solar power generation affected by the environment, the Markov decision process MDP model obtained by the model can focus on the power output fluctuation of each distributed power generation unit, and provide an efficient and stable distribution network switch automatic control optimization strategy.

2. The present invention establishes a Distflow power flow optimization constraint model, and the Distflow power flow optimization constraint model can solve the problems of distribution network optimization reconstruction, fault reconstruction, etc. Since the distribution network reconstruction can achieve the purpose of optimizing power flow distribution and improving power supply reliability and economy by selecting the power supply path of the user, the Markov decision process MDP model established by the Distflow power flow optimization constraint model can solve the distribution system failure problem and improve the reliability of the distribution system. On the basis of the MDP model, the optimal strategy for each time interval can be dynamically formulated according to the real-time state and state transition probability.

3. The present invention adopts the approximate dynamic programming (ADP) algorithm to solve the above-mentioned MDP model, which can solve the "dimensionality curse" problem.

4. The present invention aims to optimize and solve the problems of operating fluctuations, failures and malfunctions of distributed power generation units in the distribution network, improve the power supply reliability of the distribution system, and enhance the investment benefits of the distribution system.

Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for automatically controlling switches in a distribution network based on reinforcement learning theory according to an embodiment of the present invention.

FIG2 is an overall framework diagram of a system corresponding to a distribution network switch automatic control method based on approximate dynamic programming according to an embodiment of the present invention.

FIG3 is a flow chart of an approximate dynamic programming algorithm according to an embodiment of the present invention.

FIG4 is a flowchart of an offline calculation process according to an embodiment of the present invention.

FIG5 is a flowchart of an online calculation process according to an embodiment of the present invention.

FIG6 is an overall flow chart of a distribution network switch automatic control method based on reinforcement learning theory according to an embodiment of the present invention.

FIG7 is a structural block diagram of a distribution network switch automatic control system based on reinforcement learning theory according to an embodiment of the present invention.

DETAILED DESCRIPTION

The present embodiment is described in detail below, and examples of the embodiment are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

The following describes the distribution network switch automatic control method and system based on reinforcement learning theory of this embodiment with reference to the accompanying drawings.

FIG1 is a flow chart of a method for automatic control of distribution network switches based on reinforcement learning theory provided by an embodiment of the present invention. As shown in FIG1 , the method includes:

Step S1: Establish a Distflow power flow optimization constraint model and determine a reinforcement learning algorithm for approximate dynamic programming.

The step S1 comprises:

Step S11: Establish a Distflow power flow optimization constraint model according to the distribution network topology constraints and the basic theory of distribution network power flow calculation.

Specifically, the network node structure of the distribution network is mainly "tree", "radial", and radial structures, supplemented by ring structures. It is widely used in light and medium density load areas. Considering both economic and management aspects, the "radial" distribution network structure is mostly adopted in the power grid.

Assuming that a distribution network has n nodes and m connecting lines, the basic judgment equation of the "tree" structure of the distribution network is:

m＝n-1 (1)

The above formula describes the basic requirements of the "tree" structure, but the spanning tree also needs to meet connectivity. A "tree" structure with connectivity is called a spanning tree. In the spanning tree, it is necessary to meet the requirement that all nodes except the root node (substation node) have only one parent node. This requirement can be achieved using the following equation:

β _ij + βj _i = α _l , l = 1, 2,..., m (2)

∑ _j∈N(i) β _ij ＝1, i＝1, 2,..., n (3)

β _0j = 0, j∈N(0) (4)

β _ij ∈{0，1} (5)

0≤α _l ≤1 (6)

By introducing two binary variables β _ij and β _ji to correspond to each connection line in the "tree" structure, β _ij = 1 indicates that node j is the parent node of node i, otherwise β _ij = 0. ∑ is a summation function. N(i) represents the set of all nodes connected to node i. In addition, the connection state (connected or disconnected) of any two nodes in the network is represented by the variable α _l or α _ij , which can ensure that the distribution network corresponds to the spanning tree connected to the main substation. The above formulas respectively indicate that line l actually exists in the spanning tree; each node except the root node is constrained to have and only have one parent node; the substation node, i.e. the root node, has no parent node. The above five equations can ensure the connectivity of the "tree" structure, making it a spanning tree to simulate the structure of the distribution network.

The automatic control method of distribution network switches should meet the operating constraints of the distribution network within each decision time t, including network topology constraints, power balance constraints, power flow constraints, voltage limit constraints and line capacity constraints. The theoretical basis for meeting the above constraints is the DistFlow distribution network flow calculation model. The flow calculation of the power system can use the physical structure of each node in the network system, voltage phasor, active and reactive power distribution, line loss and other parameters as operating conditions to determine the operating status of the entire power system.

According to Kirchhoff's voltage law, the law of conservation of energy and Ohm's law, we can get the following formula:

_S1 ＝ _S0 - _Sloss1 - _SL1 (7)

V ₁ ∠θ＝V ₀ −z ₁ I ₀ (9)

表示。V₁∠θ表示节点1上的电压相量，与节点0处电压相量的关系可用V₁∠θ＝V₀-z₁I₀表示，I₀为线路电流，θ为相位角。Wherein, _Si ,i=1,2,...,n represents the injected power of node i. For example, the injected power of node 0 is _S0 = _P0 + _jQ0, i.e., the injected power _S0 is equal to the complex sum of the injected active power _P0 and the injected reactive power _Q0 . _Sloss1 represents the energy loss on the line from node 0 to node 1, and S _L1 represents the load on node 1. The injected power on node 1 is equal to the injected power on node 0 minus the line loss power and the load demand power. _z1 = _r1 + _jx1 represents the line impedance value connecting node 0 to node 1, _r1 and _x1 are line impedance variables. According to Ohm's law, the relationship between impedance and energy loss power can be expressed as

_{V 1} ∠θ represents the voltage phasor at node 1, and its relationship with the voltage phasor at node 0 can be expressed as V ₁ ∠θ＝V ₀ -z ₁ I ₀ , where I ₀ is the line current and θ is the phase angle.

According to the power calculation formula, the following formula is obtained:

为S₀的共轭复数，接下来再将z₁＝r₁+jx₁、

和上式代入V₁∠θ＝V₀-z₁I₀可得到下式：in,

is the conjugate complex number of S ₀ , and then z ₁ = r ₁ + jx ₁ ,

Substituting the above formula into V ₁ ∠θ＝V ₀ -z ₁ I _0, we can get the following formula:

V ₁ ∠θ＝V ₀ -(r ₁ +jx ₁ )(P ₀ -jQ ₀ )/V ₀ (11)

Taking the modulus of the phasor on both sides of the above equation, we get the following equation:

After simplification, we can get the following formula:

Based on the above analysis and using the above formula, we can extrapolate to i to get the recursive equation, and then get the following DistFlow equation:

为j节点的电功率负荷总消耗。其中，p_j和q_j分别为节点j的注入有功功率和注入无功功率，

和

分别表示j节点的电功率负荷有功功率和无功功率；

和

分别表示j节点接入发电机或者DG输出供电的有功功率和无功功率，v_j为节点j的电压，r_j和x_j为线路阻抗变量。需要特别注意的是，仅在j节点具有发电机或者DG发电单元的情况下才存在

和

这两个物理量。In order to meet the actual situation of power operation, the lowercase letters are introduced in the above formula.

is the total power load consumption of node j. Where p _j and q _j are the injected active power and injected reactive power of node j respectively.

and

They represent the active power and reactive power of the electrical power load of node j respectively;

and

They represent the active power and reactive power of the generator or DG output power supply connected to node j, _vj is the voltage of node j, _rj and _xj are line impedance variables. It should be noted that only exists when there is a generator or DG power generation unit at node j.

and

These two physical quantities.

Now we have obtained the nonlinear Distflow equation. In order to better apply the above formula to practical research work, the following two assumptions are proposed:

Assumption 1: The value of the nonlinear term in the power distribution model is very small and can be considered as zero.

Assumption 2: Assume that V _j ≈V ₀ , then we can get the following formula:

Based on the above two assumptions, the nonlinear Distflow equation obtained above can be transformed into a linear system of equations:

So far, the linearized Distflow power flow calculation model has been obtained. Its automatic control of each branch switch in the distribution network is essentially the planning and design of the topological structure of the distribution network, and the above analysis and research are based on the linearized Distflow power flow calculation model.

Based on the Distflow power flow calculation model, the following power flow optimization constraint model is considered:

min∑p _sd，j (21)

和

分别表示i节点接入发电机或者DG输出供电的有功功率和无功功率，P_d，i和q_d，i分别表示i节点处的有功、无功负荷需求，

表示任意节点，

和

分别表示P_ji和Q_ji的最大值，

和

分别表示V_i最小值和最大值；利用负荷的功率因数tanβ来表示p_sd，i和q_sd，i的关系。利用上述潮流优化约束模型的约束关系，对于n个节点的配电网络，在假设已经知道根节点即1号节点的电压V₁和各节点的负荷需求P_d，i、配电网络拓扑结构和各支路阻抗r_ji+x_ji的情况，对各节点电压V_i、V_j、注入负荷的有功功率P_ji和Q_ji以及有功切负荷和无功切负荷p_sd，i和q_sd，i等物理量进行优化求解。Among them, the new physical quantities p _sd,i and q _sd,i are introduced to represent the active load shedding and reactive load shedding at node i, respectively. Load shedding is also called load reduction, that is, reducing the load. Under normal circumstances, when encountering line failures or natural disasters, in order to maintain the power balance and stability of the power system, the behavior of disconnecting part of the load from the power grid is called load shedding. In addition, _Pik and _Qik represent the active power and reactive power flowing from node i to any node k, respectively.

and

They represent the active power and reactive power of the generator or DG output power supply connected to node i, respectively. P _d,i and q _d,i represent the active and reactive load demands at node i, respectively.

represents any node,

and

Respectively represent the maximum values of P _ji and Q _ji ,

and

Respectively represent the minimum and maximum values _{of Vi} ; use the load power factor tanβ to represent the relationship between _psd,i and qsd _,i . Using the constraint relationship of the above power flow optimization constraint model, for a distribution network with n nodes, assuming that the voltage _V1 of the root node, i.e., node 1, and the load demand _Pd,i of each node, the topological structure of the distribution network, and the impedance of each branch _rji + _xji are known, the physical quantities such as the voltage V _i , V _j of each node, the active power _Pji and _Qji of the injected load, and the active load shedding and reactive load shedding _psd,i and qsd _,i are optimized and solved.

The on/off status of some lines in the distribution network can be controlled, so the following Distflow power flow optimization constraint model can be obtained:

min∑p _sd，j (30)

The variable μ _ji is introduced in the above formula to represent the on/off state of the line. If μ _ji = 1, it means the line is closed, otherwise μ _ji = 0.

Network reconstruction is the essence of the switch automatic control strategy. It means that under the normal operation of the network, by changing the combination state of the section switch and the tie switch, that is, selecting the power supply path of the user, the purpose of optimizing the power flow distribution and improving the reliability and economy of power supply is achieved. The above model is used to solve the problems of distribution network optimization reconstruction and fault reconstruction.

Step S12: Determine a reinforcement learning algorithm for approximate dynamic programming based on reinforcement learning theory and Bellman's optimal equation.

Reinforcement learning is a type of problem in the field of machine learning that aims to enable an agent to take the best action to maximize benefits. It is used to find the best action to take in a certain environment. The difference between reinforcement learning and supervised learning is that in supervised learning, the training data (Train data) has a label value (Label), so the model itself is trained with the correct answer; in reinforcement learning, although there is no label, the agent can gradually learn the optimal action or path from repeated experience and mistakes.

Dynamic Programming (DP) originated from the fields of engineering and finance, which tend to focus on continuous states and decision control. In contrast, in the field of artificial intelligence, DP mainly involves discrete states and decisions. It is an algorithm with model-based characteristics in reinforcement learning. Under this reinforcement learning algorithm, the environment and model are known to the agent, and the optimal strategy can be learned under a given complete Markov decision process model, so it is classified as a model-based method.

There are many high-dimensional problems in DP, such as the one studied in this paper, that are often studied using tools from programming. Much of this work focuses on deterministic problems using tools such as linear, nonlinear, or integer programming.

Like other reinforcement learning algorithms, the core idea of DP is still to find the optimal decision based on the state-value function. However, the state-value function under the DP algorithm is based on the Bellman optimal equation, on which we can get:

为求期望值。利用概率论中期望的概念，可以继续进行改写：Among them, v _* ( _St ) with an asterisk subscript represents the state-value function that satisfies the value maximization, _Rt is the immediate reward at time t, γ is the discount factor, γ∈[0,1], the degree to which γ is close to 1 indicates the importance of future benefits; conversely, the degree to which γ is close to 0 indicates the importance of current benefits, _Gt+1 is the sum of all rewards until time t+1, _St is the state set, s is the state at time t, _At is the action set, a is the action at time t,

To find the expected value, we can continue to rewrite it using the concept of expectation in probability theory:

v _* (S _t )=max _a ∑ _{s′, r} p (s′, r|s, a) [r+γG _t+1 ] (40)

Among them, max is the maximum value function, s′ is the state at the next moment, r is the reward value of the environment feedback, and p(s′, r|s, a) is the probability of the environment feedback reward r and the transition to the next moment state s′ under the premise of state s and action a.

It should be noted that G _t+1 and v _* (S _t+1 ) are interchangeable, and we can obtain:

v _* (S _t ) = max _a ∑ _{s′, r} p (s′, r|s, a) [r+γv _* (S _t+1 )] (41)

v _* (S _t )=max _a {R _t +∑ _{s′, r} p(s′, r|s, a)*[γv _* (S _t+1 )]} (42)

The equation after substitution transforms the Bellman equation into a recursive update equation that approximates the ideal value function, which is the prototype of the DP algorithm. Based on the above equation, all dynamic programs can be written in a recursive way. The recursive process links the value of a specific state v _t (S _t ) at a certain time t with the value of the state v _t+1 (S _t+1 ) at the next time t′.

However, the DP algorithm has three "curses" in terms of dimension, which makes the solution process complicated, mainly manifested in the following three aspects: the state space S is too large to calculate the value function v _* ( _St ) within an acceptable time; the decision space A is too large, there are too many permutations and combinations of actions, which often increase exponentially, and the optimal action cannot be found quickly; the result space is too large to calculate the expected value of future rewards.

转化成概率分布的形式后得到∑_s′，rp(s′，r|s，a)[r+γv_*(S_t+1)]。观察该式发现MDP(MarkovDecision Process，马尔可夫决策过程)的状态过多会导致求解缓慢或无法求解的问题，提出近似动态规划算法来解决，将状态-价值函数v_*(S_t)作近似处理并引入了一个新的概念：Approximate dynamic programming is based on an algorithmic strategy that progresses step by step over time, so it is also called forward dynamic programming. The DP algorithm needs to solve

After being converted into a probability distribution, we get ∑ _{s′, r} p(s′, r|s, a)[r+γv _* (S _t+1 )]. By observing this formula, we find that too many states in the MDP (Markov Decision Process) will lead to slow or unsolvable problems. We propose an approximate dynamic programming algorithm to solve this problem, approximate the state-value function v _* (S _t ) and introduce a new concept:

The post-decision state refers to the state of the system after a decision is made and before any new information arrives. This is a state between S _t and S _t+1 after a decision a _t is made, denoted by

After proposing this concept, the above formula can be rewritten as:

替代的是求解未来奖励的期望值。

Instead, we solve for the expected value of future rewards.

Step S2: Obtain historical output data of photovoltaic and wind power distributed generation units, and establish a distributed generation unit output power fluctuation and conversion model based on the historical output data.

In step S2, before establishing the output power fluctuation and conversion model of the distributed generation units, the method also includes: analyzing and quantifying the output power fluctuation and uncertainty of each distributed generation unit, so as to use the quantified value as one of the state inputs of the Markov decision process MDP model of the distribution network.

According to relevant historical data, the active power output of wind farms and photovoltaic farms on a long-term scale is highly random, which meets the requirements of exploring the volatility of new energy power generation units such as wind power generation and photovoltaic power generation with the natural environment.

The volatility and randomness of wind farm active output are affected by many objective factors, such as the geographical and climatic conditions of the area where the wind farm is located; the spatial distribution and arrangement of wind turbines. Using the wind power active output data with a sampling period of 15 minutes, the average daily output of wind power is calculated as follows:

Among them, P _{daily average output} and W _day represent the daily average output and daily power generation of wind power generation respectively, P(t) is the active output at time t, and the daily average output of the wind farm is analyzed according to the above formula. The active output of the wind farm is affected by the natural weather conditions in different seasons, and the fluctuations within a year are very violent. Therefore, in order to qualitatively select the active power output fluctuation data of wind power DG, the output data of the general output day of the wind farm (daily power generation = annual power generation/365day) is selected. The peak output of the general output day is from 3 pm to 10 am and from 10 pm to 10 am, and the rest of the time is approximately equal to no output.

The volatility and randomness of the active output of the photovoltaic power field are mainly caused by the sunshine hours, altitude, and natural disasters (drought, rainstorm, frost) of the photovoltaic power plant. Similarly, the actual active output data of photovoltaic power generation with a sampling period of T=15min from the general output day is used. Weather conditions directly lead to drastic fluctuations in the active output of photovoltaic power stations; the output in clear weather is much better than that in rainy weather; photovoltaic DG has intermittent output, and there will be a long period of continuous suspension during the night.

In this embodiment, after the distributed power generation unit output power fluctuation and conversion model is established, the distributed power generation unit output power fluctuation and conversion model can be used to simulate the volatility and uncertainty of the output power of each distributed power generation unit.

Specifically, the quantified value of the output power fluctuation of each distributed generation unit (DG) and the corresponding distribution network topology are regarded as the state of MDP, while the power generation output power of DG has high uncertainty.

DG has k output levels in each time period. The larger the k value, the more precise the discrete quantization of the DG output power simulation and the smaller the error. The probability of an output level k at time t to another output level k′ at time t+1 is represented by ∏ _kk′. This conversion probability can be presented by Monte Carlo simulation of historical data of wind power DG. For example, the number of historical occurrences of output level k at time t is m times, and the number of conversions from output level k to output level k′ at time t+1 is n times, so ∏ _kk′ is equal to n/m.

Step S3: Determine the topological structure of the distribution network, obtain the active output data of the distributed generation units, the status information of the controllable section switches and the interconnection switches of the distribution network, and the calculation result information of the Distflow power flow optimization constraint model, and establish the distribution network Markov decision process MDP model according to the output power fluctuation and conversion model of the distributed generation units, the topological structure of the distribution network and the information obtained in step S3.

Among them, the overall framework diagram of the system corresponding to the automatic control method of the distribution network switch based on approximate dynamic programming is shown in Figure 2. Figure 2 consists of two parts: reinforcement learning algorithm and distributed distribution network. At each moment of the operation of the distributed distribution network, external conditions such as weather and natural disasters and network topology are input into the MDP process as state parameters _St, that is, external environmental states. The specific manifestation is that external conditions such as weather and natural disasters affect the output of photovoltaic power generation and wind power generation, and then the switch strategy _At is output. According to the pre-set relevant parameters, such as network line operation cost, load shedding, etc., the reward value at the current moment is fed back, that is, the immediate reward _Rt is fed back. The reinforcement learning ADP algorithm uses the post-decision state and forward dynamic algorithm, and iterates the calculation based on the above data, the real-time operation status of the distribution network and the accumulated reward value. The calculation results of the algorithm will be fed back to the intelligent agent. Each complete time cycle is a training. After hundreds of trainings, each decision will get a converged calculation value, which will be stored in a table. Finally, the size of these values is compared to find the optimal strategy at each moment.

The step S3 of establishing the Markov decision process MDP model of the power distribution network includes:

Step S31: Modeling the output power fluctuation status quantization value of each distributed power generation unit and the corresponding distribution network topology structure as state parameters of the distribution network Markov decision process MDP model.

State set S _i,t : The switches on each line in the distribution network topology are defined in turn as the network topology:

Ξ _t = [swt ₁ , swt ₂ , swt ₃ ,..., swt _n ] (45)

Wherein, swt _n represents the current state of switch n, and each switch has two states, open and closed, represented by binary numbers 0 and 1 respectively. Therefore, the current distribution network topology Ξ _t can be represented by an n-bit binary number. In the Markov decision model of the distribution network, the network topology and the power output level of the distributed generation unit at time t are defined as a state set:

S _i,t =[Ξ _t |k _1,t ,k _2,t ,k _3,t ,...,k _dg,t ,...,k _DG,t ] (46)

Wherein, Ξ _t represents the distribution network topology at time t, k _dg,t represents the DG power output level at time t.

Step S32: Modeling the action states of switches connecting the branches of the power distribution network as action combination parameters of the Markov decision process MDP model of the power distribution network.

The action state of the switch includes the switch changing the switch state at the current moment and maintaining the switch state at the current moment.

Action set A _t (S _{i, t} ): The switches [swt ₁ , swt ₂ , swt ₃ , ..., swt _n ] connecting each node have and can only have two actions at each time t, that is, changing the switch state at time t-1 (converting from closed state to open state, from open state to closed state) or maintaining the switch state at time t-1, which is represented by the action a _swt1 = a ₁ or a ₂ in a single switch state. Each switch in the tree network line has the above two actions at any time, and A _t (S _{i, t} ) can be expressed as:

A _t (S _{i, t} )=[a _swt1 , a _swt2 ,..., a _swti ] (47)

Among them, a _swti represents the action in the i-th switch state, and S _i,t represents the state set.

Step S33: Select load shedding and line operation cost modeling in the Distflow power flow optimization constraint model considering distributed generation failure as reward function reference indicators of the distribution network Markov decision process MDP model.

Instantaneous reward function R(S _{i, t} , a _t ): When a specific action a _t ∈ _{A t} (S _{i, t} ) is applied at time t, the state of the network will change from s _t ∈ S _{i, t} to s _t+1 ∈ _{S i, t} , where s _t and s _t+1 represent the states at time t and t+1, respectively, and the system observes an instantaneous reward value R(S _{i, t} , a _t ), which is measured according to the network flow calculation of the entire distribution system.

Specifically, the load shedding in the distribution network flow calculation model Distflow considering distributed generation failures can be selected as a reference indicator of the reward function R(S _{i, t} , a _t ). R(S _{i, t} , a _t ) represents the cost of load shedding and the line operation cost in the distribution system. Its value is always negative. To maximize the value of R(S _{i, t} , a _t ), the cost of load shedding must be smaller. The size of load shedding reflects the input and output power balance and system stability of the distribution system. The smaller its value, the more it can meet the working requirements and economic budget of the distribution network.

Step S34: define the state transition probability of the Markov decision process MDP model of the distribution network so as to take into account the uncertainty caused by the probability of change of the power output level of each distributed generation unit.

In this embodiment, the uncertainty caused by the probability of change of the DG power output level can be considered to define the state transition probability of the Markov decision process MDP model in which the state is converted to the state under the action of the action.

State transition probability P( _{Sj, t+1} |Si _{, t} , _at ): Considering the uncertainty brought by the probability _∏kk′ of the change of DG power output level from time t to time t+1, the state transition probability P(Sj _{, t+1} |Si, _t , _at ) of state _{Si, t} transforming to state Sj _{, t+1} under the action _at can be expressed as follows:

P(S _{j, t+1} |S _{i, t} , a _t )=∏ _kk′ ×P(Ξ _t+1 |Ξ _t , a _t ) (48)

Where P(Ξ _t+1 |Ξ _t , a _t ) represents the probability that the distribution network topology Ξ _t is converted to Ξ _t+1 under the action a _t at time t. Since Ξ _t is changed to Ξ _t+1 due to network reconstruction, and the distribution network topology conversion is certain under a specific reconstruction operation, this probability is "100%" or "0%". ∏ _kk′ is the probability of an output level k at time t to another output level k′ at time t+1 when there is only one DG; if there are n DGs, the conversion probabilities of the power output levels of each DG should be multiplied, that is, the above formula should be written as a continuous multiplication of n ∏ _kk′ .

Step S4: Use the reinforcement learning algorithm of approximate dynamic programming to solve the Markov decision process MDP model of the distribution network, and output the optimal strategy for automatic control of distribution network switches in real time.

In the step S4, before outputting the optimal strategy for automatic control of the distribution network switches in real time, the method further comprises: performing offline iterative training on the Markov decision process MDP model of the distribution network.

The steps of offline iterative training of the Markov decision process MDP model of the distribution network include: inputting the active output data of the distributed generation units, outputting the automatic control strategy of the distribution network switches, and feeding back the evaluation indicators of the real-time operation of the distribution network.

According to the distribution network Markov model and Bellman optimal equation established in step S3, and taking into account the uncertainty and volatility of DG output, the dynamic characteristics of the entire system are given by the four-parameter probability distribution P(S _{j, t+1} |S _{i, t} , a _t ). The recursive optimized state-value function can be expressed as follows:

Among them, γ is the discount factor, γ∈[0,1], the degree to which γ is close to 1 indicates the importance of future benefits; conversely, the degree to which γ is close to 0 indicates the importance of current benefits, r(S _{i, t} , a _t ) is the immediate reward value of environmental feedback under the current state S _{i, t} and action a _t , and v _t+1 (S _{j, t+1} ) is the state-value function value of the next state S _{j, t+1} . Let γ＝1 and put r(S _{i, t} , a _t ) out of brackets, and we get the following formula:

表示在当前时刻状态S_i，t和动作a_t前提条件下，下一时刻状态S_j，t+1的状态-价值函数的期望值，即时收益函数R(S_i，t，a_t)由两部分组成：配电系统中切负荷的成本和线路运营成本，其值为恒负。故R(S_i，t，a_t)可以表示为：in,

It represents the expected value of the state-value function of the next state _Sj,t+1 under the current state S _i,t and action a _t . The instant benefit function R(S _i,t , a _t ) consists of two parts: the cost of shedding load in the distribution system and the line operation cost, and its value is always negative. Therefore, R(S _i,t , a _t ) can be expressed as:

R(S _{i, t} , a _t )=∑ _b∈B (c _b *p _{sd, b} )+∑ _l∈L (c ₁ *μ _{b, b′} ) (52)

将

定义为

从而实现了对数学期望求解过程的形式转换，得到下列式：Where B is the node set, c _b is the load shedding cost coefficient, p _sd,b is the active load shedding at node b, c ₁ is the line operation cost coefficient, μ _b,b′ is the line operation cost from node b to node b′. After introducing the concept of the post-decision state, S _i,t , _at is rewritten as the post-decision state variable

Will

Defined as

This achieves the formal transformation of the mathematical expectation solution process and obtains the following formula:

表示决策后状态

的状态-价值函数值，

表示在上一时刻状态S_j，t-1和动作a_t-1前提条件下，当前时刻状态S_i，t的状态-价值函数的期望值，R_t+1(S_j，t+1，a_t+1)表示下一时刻状态S_j，t+1和动作a_t+1下环境反馈的即时奖励值，

表示决策后状态

的状态-价值函数值，上式将多周期和大规模的基于MDP的随机模型转换为每个决策周期内每个状态的单周期确定性模型并可以通过迭代来求解

in,

Indicates the state after decision

The state-value function value of

represents the expected value of the state-value function of the current state S _{i, t} under the premise of the previous state S _{j, t-1} and action a _t-1 , R _t+1 (S _{j, t+1} , a _t+1 ) represents the immediate reward value of the environmental feedback under the next state S _{j, t+1} and action a _t+1 ,

Indicates the state after decision

The state-value function value of the above formula converts the multi-period and large-scale MDP-based stochastic model into a single-period deterministic model for each state in each decision cycle and can be solved by iteration

Introducing the variable n to describe the nth iteration, we get the following formula:

表示第n次迭代中状态S_i，t的状态-价值函数值，

表示第n次迭代中决策后状态

的状态-价值函数值，结合决策后状态变量的状态-价值函数递推形式，再利用前向动态算法来逐轮次更新

in,

represents the state-value function value of state _Si,t in the nth iteration,

Represents the state after decision in the nth iteration

The state-value function value of the decision-making state variable is combined with the state-value function recursive form, and then the forward dynamic algorithm is used to update the state-value function round by round.

表示第n次迭代中决策后状态

的状态-价值函数值，

表示第n-1次迭代中决策后状态

的状态-价值函数值，

表示第n次迭代中状态S_j，t的状态-价值函数值，系数α是一个小于1的平滑参数，依托这个迭代更新公式以及足够多的迭代次数，则每一个决策后状态的状态-价值

都可以得到一个对应的收敛值。并可以得到每一个马尔可夫状态下不同开关动作的价值，最后通过比较这些价值的大小来找到最佳动作和决策。in,

Represents the state after decision in the nth iteration

The state-value function value of

Represents the state after decision in the n-1th iteration

The state-value function value of

represents the state-value function value of state S _j,t in the nth iteration. The coefficient α is a smoothing parameter less than 1. Based on this iterative update formula and a sufficient number of iterations, the state-value of each state after decision making is

A corresponding convergence value can be obtained. And the value of different switch actions in each Markov state can be obtained. Finally, the best action and decision can be found by comparing the sizes of these values.

The specific pseudo code of the ADP algorithm is as follows:

Step 1 Initialization.

Step 1a sets the network topology state and DG fluctuation output state.

和

状态-价值函数表。Step 1b Initialization

and

State-value function table.

Step 1c sets the step size α∈(0, 1] and the number of iterations N.

Step 2 Do when t = 1, 2, ..., T:

其中，要让代入a_t来解决最大值问题。Step 2a: Use Gurobi mathematical programming optimizer to solve R(S _{i, t} , a _t ) using mixed integer linear programming and obtain

Among them, we need to substitute a _t to solve the maximum value problem.

Step 2b uses the forward dynamic algorithm to update

判断t是否小于T，并在不小于T时，执行下述步骤；Step 2c: Get the post-decision state from the state Si _,t and action _at

Determine whether t is less than T, and if it is not less than T, perform the following steps;

进入下一时刻马尔可夫状态S_j，t+1。Step 2d: According to the DG fluctuation and network topology, the decision-making state

Enter the next Markov state S _j,t+1 .

Step 3 makes n=n+1. Determine whether n is less than or equal to the number of iterations N. If so, repeat step 2. Otherwise, end the loop and go to step 4.

以及其对应的马尔可夫状态S_i，t。Step 4 returns the convergence value

And its corresponding Markov state Si _,t .

The flow chart of the approximate dynamic programming algorithm is shown in Figure 3.

近似动态规划算法即可实现这一目标。离线计算部分的输入信息包括配电系统网络拓扑情况和DG输出功率的不确定性和波动性。将这些信息送入ADP算法中，返回得到

它是一个使得

反复迭代并最终收敛的多周期过程。The learning and training process of the MDP model for automatic control of distribution network switches under DG output power fluctuation is an iterative process of offline calculation. Figure 4 shows the flow chart of offline calculation. The purpose of offline calculation is to obtain the convergence value of the state value after decision making.

Approximate dynamic programming algorithm can achieve this goal. The input information of the offline calculation part includes the distribution system network topology and the uncertainty and volatility of DG output power. This information is fed into the ADP algorithm and returned

It is a

A multi-cycle process that iterates repeatedly and eventually converges.

In step S4, after outputting the optimal strategy for automatic control of the distribution network switches in real time, the method further comprises: displaying the decision result textually and the real-time distribution network topology structure on the human-computer interaction interface.

After the training and learning of the MDP model for automatic control of distribution network switches under DG output power fluctuations, it is necessary to automatically predict the optimal switching strategy. The prediction stage and decision stage are an online calculation process, and the online calculation flow chart is shown in Figure 5.

和每个时间段观察到的马尔可夫状态S_i，t作为单周期确定性模型的输入，目标函数仍是R(S_i，t，a_t)，其约束条件是步骤S1提出的配电网络结构与潮流约束条件，然后返回

和最佳动作A_t(S_i，t)。在这种情况下，在线计算过程可以在每个决策时刻获得基于实时马尔可夫状态S_i，t的最大价值策略。The role of online calculation is to obtain the optimal switching strategy of the distribution network based on the real-time Markov state S _j,t observed by the agent at each moment. As shown in Figure 5, the post-decision state value output in offline calculation is

The Markov state Si _,t observed in each time period is used as the input of the single-period deterministic model. The objective function is still R(Si _,t ,a _t ), and its constraints are the distribution network structure and power flow constraints proposed in step S1, and then return

and the best action A _t (S _{i, t} ). In this case, the online calculation process can obtain the maximum value strategy based on the real-time Markov state S _{i, t} at each decision moment.

FIG6 is an overall flow chart of the automatic control method of the distribution network switch based on the reinforcement learning theory of the present invention. As shown in FIG6, it mainly consists of four parts: derivation of relevant theoretical knowledge, establishment of the automatic control MDP model of the distribution network switch under DG fluctuation, solution of the MDP model and collation of analysis results, and completion of the human-computer interaction APP. The theoretical derivation part derives the Bellman equation based on the ideas of reinforcement learning and Markov decision model, and introduces the basic principles of dynamic programming and approximate dynamic programming on this basis. The establishment of the MDP model considers the volatility of new energy power generation units such as wind power generation and photovoltaic power generation with the natural environment, and establishes a Markov decision model for automatic control of distribution network switches. First, a conversion fluctuation model between different output levels of distributed power generation units is established, and then a distribution network MDP model is established according to the MDP model idea in the reinforcement learning theory, and finally the corresponding program is written according to the structural constraints of the distributed distribution network and the Distflow flow calculation constraints. Solving the MDP model targets the problems and difficulties that will occur when the traditional reinforcement learning algorithm solves the MDP model, and proposes an automatic control ADP algorithm for distribution network switches to solve this problem. The core method is to introduce the post-decision state and forward dynamic algorithm. We strive to achieve the transformation of multi-cycle and large-scale MDP-based random models into single-cycle deterministic models for each state in each decision cycle. Finally, we write a human-computer interaction APP (Application) to enable users to interact with the distribution network switch automatic control software system and directly view the text and picture results of the optimal decision.

Fig. 7 is a block diagram of a distribution network switch automatic control system based on reinforcement learning theory according to an embodiment of the present invention. As shown in Fig. 7, the distribution network switch automatic control system 100 based on reinforcement learning theory includes an establishment module 10, a determination module 20, an acquisition module 30 and a calculation module 40, wherein the establishment module 10 is connected to the determination module 20, the acquisition module 30 and the calculation module 40 respectively, and the calculation module 40 is also connected to the determination module 20.

Among them, the establishment module 10 is used to establish the Distflow flow optimization constraint model. The determination module 20 is used to determine the reinforcement learning algorithm of approximate dynamic programming. The acquisition module 30 is used to obtain the historical output data of photovoltaic and wind power distributed generation units, so that the establishment module 10 can establish the output power fluctuation and conversion model of the distributed generation unit according to the historical output data. The establishment module 10 is also used to establish the distribution network Markov decision process MDP model according to the distribution network topology determined by the determination module 20, the active output data of the distributed generation unit acquired by the acquisition module 30, the controllable segmented switches of the distribution network, the interconnection switch status information and the calculation result information of the Distflow flow optimization constraint model, and the output power fluctuation and conversion model of the distributed generation unit. The calculation module 40 is used to solve the distribution network Markov decision process MDP model according to the reinforcement learning algorithm of approximate dynamic programming, and output the optimal strategy for automatic control of the distribution network switches in real time.

It should be noted that the specific implementation of the distribution network switch automatic control system based on reinforcement learning theory in the embodiment of the present invention can refer to the specific implementation of the distribution network switch automatic control method based on reinforcement learning theory mentioned above. To avoid redundancy, it will not be repeated here.

In summary, the present invention proposes a distribution network switch automatic control method and system based on reinforcement learning theory. The present invention can fully consider the working efficiency volatility of renewable energy such as wind power generation and solar power generation, and establish a Markov decision process MDP model for automatic control of distribution network switches. The model combines the general model of distribution network configuration with the power supply reliability rate, focuses on the power output fluctuation of each distributed power generation unit, and can provide an efficient and stable distribution network switch automatic control optimization strategy; the present invention uses the MDP process to improve economy, reliability, reduce losses, and balance loads. The main purpose completes the network reconstruction of the distribution network containing distributed power generation units, and on the basis of the model, the optimal strategy in each time interval is dynamically formulated according to the real-time state and state transition probability; the present invention adopts an approximate dynamic programming ADP algorithm to solve the above-mentioned MDP model, which can solve the "dimensionality curse" problem; the present invention aims to optimize and solve the working fluctuation, failure, and fault problems of distributed power generation units in the distribution network, improve the power supply reliability of the distribution system, and improve the investment benefit of the distribution system.

It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims (10)

1. The utility model provides a distribution network switch automatic control method based on reinforcement learning theory which is characterized by comprising the following steps:

step S1: establishing a Distflow tide optimization constraint model, and determining a reinforcement learning algorithm approximating dynamic programming;

step S2: acquiring historical output data of a photovoltaic and wind power distributed generation unit, and establishing a distributed generation unit output power fluctuation and conversion model according to the historical output data;

step S3: determining a distribution network topology structure, acquiring active output data of a distributed power generation unit, controllable sectionalizing switches of a power distribution network, contact switch status information and Distflow power flow optimization constraint model calculation result information, and establishing a distribution network Markov decision process MDP model according to the output power fluctuation and conversion model of the distributed power generation unit, the distribution network topology structure and the information acquired in the step S3;

step S4: and solving the MDP model of the power distribution network Markov decision process by adopting the reinforcement learning algorithm of the approximate dynamic programming, and outputting the automatic control optimal strategy of the power distribution network switch in real time.

2. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 1, wherein the step S1 comprises:

Step S11: establishing the Distflow power flow optimization constraint model according to the power distribution network topological structure constraint and the power distribution network power flow calculation basic theory;

step S12: and determining the reinforcement learning algorithm which approximates to the dynamic programming according to reinforcement learning theory and a Belman optimal equation.

3. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 1, wherein in the step S2, before the step of establishing the output power fluctuation and conversion model of the distributed power generation unit, the method further comprises: the output power fluctuations and uncertainty conditions of each distributed generation unit are analyzed and quantified so as to take the quantified values as inputs to the power distribution network markov decision process MDP model.

4. The method for automatically controlling the switching of the power distribution network based on the reinforcement learning theory according to claim 3, wherein fluctuation and uncertainty of the output power of each distributed power generation unit can be simulated through the fluctuation and transformation model of the output power of the distributed power generation unit.

5. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 4, wherein the step S3 comprises:

Step S31: modeling output power fluctuation status quantized values of each distributed generation unit and corresponding power distribution network topological structures as state parameters of a power distribution network Markov decision process MDP model;

step S32: modeling the action state of a switch connected with each branch line of a power distribution network as an action combination parameter of an MDP model of a Markov decision process of the power distribution network;

step S33: selecting cut loads in the Distflow power flow optimization constraint model considering distributed power generation faults and modeling line operation cost as a rewarding function reference index of the distribution network Markov decision process MDP model;

step S34: the state transition probabilities of the distribution network markov decision process MDP model are defined so as to take into account the uncertainty caused by the probability of change in the generated power output level of each distributed generation unit.

6. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 1, wherein in step S4, before outputting the power distribution network switch automatic control optimal strategy in real time, the method further comprises: and performing offline iterative training on the MDP model of the power distribution network Markov decision process.

7. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 6, wherein the step of performing offline iterative training on the power distribution network markov decision process MDP model comprises: and inputting active output data of the distributed power generation unit, outputting an automatic control strategy of a power distribution network switch, and feeding back an evaluation index of the real-time running condition of the power distribution network.

8. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 1, wherein in the step S4, after outputting the power distribution network switch automatic control optimal strategy in real time, the method further comprises: and displaying decision result characters and displaying the network topology structure of the real-time power distribution network on the man-machine interaction interface.

9. The method for automatically controlling a power distribution network switch based on reinforcement learning theory according to claim 5, wherein the action state of the switch comprises: the switch changes the switch state at the current time and maintains the switch state at the current time.

10. An automatic control system for a power distribution network switch based on reinforcement learning theory is characterized by comprising:

the establishing module is used for establishing a Distflow tide optimization constraint model;

The determining module is used for determining a reinforcement learning algorithm approximate to dynamic programming;

the acquisition module is used for acquiring historical output data of the photovoltaic and wind power distributed generation units so that the establishment module establishes an output power fluctuation and conversion model of the distributed generation units according to the historical output data;

the establishing module is also used for establishing an MDP model of a power distribution network Markov decision process according to the power distribution network topological structure determined by the determining module, the active output data of the distributed power generation units, the controllable sectional switch of the power distribution network, the state information of the contact switch and the calculation result information of a Distflow flow optimization constraint model, and the fluctuation and conversion model of the output power of the distributed power generation units;

and the calculation module is used for solving the MDP model of the power distribution network Markov decision process according to the reinforcement learning algorithm of the approximate dynamic programming and outputting the automatic control optimal strategy of the power distribution network switch in real time.

Images